[Duckjones]

I've read a few posts that had some good ideas for future enhancements to HALi, I've got a few of my own too. Now that 4.0 is released, I thought it might be a good idea to consolidate those ideas into a single thread. So I'll add mine, anyone else feel free to chime in.

1.) Method for changing the Unrecognized Speech Tag phrase (i.e. "Please repeat"), it would be so much fun to be able to change this on the fly. Additionally the ability to change the Timeout warning ("Are you still there?").

2.) An override for AskListContext that takes an array of strings instead of a comma delimited string of context tags.

3.) This one's a stretch, but maybe an interface into the voice recording feature found in Reminders. This is probably easily accomplished by integrating a 3rd party WAV recorder, but it might be easier if HAL could accomplish the task instead.

[JimMcGowanInlet]

[quote=Duckjones;20984]2.) An override for AskListContext that takes an array of strings instead of a comma delimited string of context tags.


Due to the interface between HALi, HALCOM and HAL Server it would be difficult to implement this suggestion.

[Duckjones]

Ok, I thought of another one that's probably an even bigger stretch, but it would improve the interactivity of HALi. Could we embed ASR codes, similar to the TTS codes, that would tell HAL to interpret a certain data type within a speech tag?

Using an example that already works within HAL, a tag might get registered as follows:

"In <int> minutes, turn on the bedroom light"

And HAL would be able to recognize that entire tag and parse the numeric value out of it to use. I'd love to be able to do something like "List movies starting between <time> and <time>" or something to that effect, it would just rock.

[JimMcGowanInlet]

I do not follow you. Where is the value of <int> comming from?

[Duckjones]

The <int> field, or <time>, <date>, <dayofweek>, <on/off>, etc etc, would be supplied when the user invoked the voice tag, the code would just tell HALi to expect the user to say something of that data type when they use this voice tag. So if I register a voice tag like:
"List movies starting between <time> and <time>"

And then tell HAL to
"List movies starting between 12 PM and 1 PM"
then HAL would be able to at least recognize the tag and spit it back to the SpeechNotify event, even better if it could store the "12 PM" and "1 PM" values in an array passed in via the event args, or maybe just put special characters around those values in the string so that the user code can parse them out.

[JimMcGowanInlet]

You may be able to do that now with the VR context. For example.

You VR, "Tell me what is on TV"

HAL TTS's back "Starting at what hour?" Then give it a custom VR context of 1,2,3,4,5,6.... Now

Then HAL TTS's "Ending when", and you throw another custom context.

Put the results into your Voice tag and you have your complete tag to send to hal.

[Duckjones]

True, that will get the job done. Although it adds more levels of complexity, especially if you're trying to do a full date and time (i.e. January 25th at 5:30pm, for example, would need to be done by month, then day, then hour, then minute), but HAL seems to handle these situations far more gracefully for it's own recognized tags.

So while trying to add similar features through HALi is doable, one of the most enjoyable features of HAL is being able to talk to it like you would a person, I think it would add an enormous amount of value to the user experience if HALi apps could retain this functionality. If it's at all possible, that is.

[JimMcGowanInlet]

I'm trying to figure out if your idea is possible. The first thing that comes to mind is VR, HAL must associate a command with an action. The difficulty is, you speak the command in its enirety, HAL would need to match the command to your template even though the spoken command would be different every time. Once it was matched to your template HAL would need to idetify the words that represent the tags and isolate them to form the complete tag.

I wonder if its possible to speak to HAL tagless. That would make it much easier to work with. Hummmmm.

[Duckjones]

That's why I was thinking awhile back of hooking in another ASR engine. The only one I've seen in action is that Dragon Naturally Speaking one, although I'm sure there's a host of others, but the basic idea would be the other app translates speech into text which then gets fed into the HALi app, which parses the text and then sends commands to HAL.

Off the top of my head I'd say that'd be a huge pain in the butt to first get HAL's attention and then trick it into ignoring anything spoken after that so the other engine could do it's thing. So having HAL do the work would be not only more cost efficient, but a much cleaner solution as well.

[VincentB]

I think DuckJone's idea of being able to have HAL recognise a phrase with tags is an excellent idea... considering HAL is already able to detect "turn on the light for x minutes", then it should already have the logic to detect a tag-inserted phrase. There's no need to make HAL support the phrase in its rules.. just fire them in a HALi event and have the 3rd party programmer do the rest.


Here's the design I'd propose:

1) Allow a programmer to register an ASR using HAL-approved tags (eg, HAL could design some HALi tags like <number>, <date>, <time>, <timemeasurement>, etc).

2) Then a programmer would register an ASR phrase as normal. Eg: "say hello to me in <number> <timemeasurement>"

3) When HAL detects a phase that matches this (eg: "say hello to me in 5 minutes"), it could simply fire a NEWLY DESIGNED speech-recognised HALi event. This new event would be identical to the normal speech-recognised event, but also provide the 'list' parameter that was passed into the ASR function. This is so when the event is received, the programmer can scan the 'list' string and figure out how they should be parsing the speech string.

4) The 3rd party programmer would then handle the new event, scan the 'list' string, figure out how to parse the 'speech' string, and then do what ever they wanted.


So here's an example:

Programmer's HALi app:
C_CASR asr = GetHali().GetAsr();
asr.CTXAddWord("sayinghello", "say hello to me in <time> <timemeasurement>", "", "");

HAL's Algorithm
* HAL detects the user saying "say hello in 5 minutes", and works out that it matches the "say hello in <number> <timemeasurement>" tagged-ASR.
* So HAL fires the new Speech Recognised event, providing the sList ("sayinhello"), the string ("say hello in 5 minutes") and the confidence (eg: 100%)

Programmer's HALi event handled:
void OnTaggedPhraseRecognised(string sList, string sSpeech, int lConfidence)
{
// step 1 - check to see if sList = "sayinghello"
// step 2 - if it does, extract the 1st word after the string "say hello to me in" .. this will be a number, which we'll store in a variable called x
// step 3 - extract the next word. This will be the time measureument
// step 4 - if the timemeasurement = "minutes", create a timer for x minutes.
// when the timer expires, call the TTS function with "hello there!"
}

All of that make sense? Now, i must admit, as I wrote this, i saw that HAli's ASR function provides 'before' and 'after' parameters, which I'm not sure what they do. They don't already do what I'm after, do they? What exactly are they for?

Anyone see any problems with my pseudocode design, for HAL to implement? It matches what DuckJones is requesting, and i think his request is very valid. It will allow programmers to make HAL amazingly sophisticated.