Natural Language control for openHAB

[GoogleCodeExporter]

1. Feature Description

openHAB needs a new interface with REST API access for processing natural 
language commands from user. This interface must provide ability to supply the 
natural language command itself (command text in natural language, like "switch 
off all lights" or "open shades on garden side" or "I'm back home") and 
additional meta/context data which should be set as a list (hash) of flexible 
parameters. Those parameters can include but are not limited to: location 
context (either global location, like GPS data, or local location, like room 
name in the house, or standing outside of the house in the garden), language 
locale (us_EN, de_DE, ru_RU, etc) which is used in command text, uniq device id 
(uuid can be one-time-generated by the client app on the device), user (can be 
taken from auth data), maybe some other information in future.
The most visible and flexible way of processing those commands for now would be 
.rules. So this interface should provide a way to supply a trigger event into 
rules, which will trigger the command processing. At the end of rule a response 
to client should be provided in natural languages, like "switching all lights 
off", or "openin shades on garden side", or "switching to at home mode", etc. 
In case no natural language response is provided by rule, interface should 
provide some generic default acknowledgment response like 'Accepted', in case 
of error in rule and no response provided it should respond with generic 
default error message like 'I have an error while processing your command'.
In future this interface can be enhanced with some natural language processing 
instead of creating dumb rules for every type of voice command :-)

2. Example Use Case
The most exciting use case is voice recognition on the UI application side. The 
app can perform voice recognition by user request (pressing some icon, for 
example) and send a text of recognized command to openhab natural language 
interface. When getting response, application can tell it back to user using 
Text-To-Speach features of the device it is running on.
Another exciting use case is continuous voice recognition on a dedicated device 
installed in user premises. Just imagine a dialog: User: 'Jarvis!' openHAB: 'At 
your command!' User: 'Switch on the lights' openHAB: 'Switching on the lights!'
Of cause this interface can be used in chat applications - XMPP control channel 
can use it instead of predefined menu-like set of commands to address openHAB 
objects.
Use cases for this interface are only limited with your fantasy!

Original issue reported on code.google.com by belovic...@gmail.com on 20 May 2013 at 8:30

[GoogleCodeExporter]

Original comment by kai.openhab on 22 May 2013 at 8:31

[GoogleCodeExporter]

What's the status of this feature request? Is anyone working on something 
similar? I started reading about Pocketsphinx and Sphinx-4 and in theory it 
shouldn't be too hard to integrate Sphinx-4 in openHAB.

I would love to call my home "Jarvis"! :)

Original comment by mishoboss on 1 Nov 2013 at 3:07

[GoogleCodeExporter]

Speech recognition engine is not a problem here, really. HABDroid (snapshot 
build) already have voice recognition built into it for example. The real 
question is a natural language interface, which will be able to interpret your 
recognised commands and phrases, so that instead of saying "Switch on item 
Outside_Lights" you could say "Switch on outside lights" or "outside lights on" 
etc and it will understand what do you want your home to do.

Original comment by belovic...@gmail.com on 1 Nov 2013 at 3:30

[GoogleCodeExporter]

That's really nice news about HABDroid. However the real thing would be if you 
place some microphones around your home and there is a continues speech 
recognition always on. This way that scenario could happen for real: "User: 
'Jarvis!' openHAB: 'At your command!' User: 'Switch on the lights' openHAB: 
'Switching on the lights!'"

Natural speech recognition sounds really good, but I'm not sure if there is a 
good enough product/service that will produce e.g. 95% success rate. Maybe in 
5-6 years :)

Original comment by mishoboss on 1 Nov 2013 at 3:48

[GoogleCodeExporter]

Original comment by teichsta on 5 Nov 2013 at 10:47

• Added labels: To-Github

[GoogleCodeExporter]

I haven't tried this idea myself but

some guy on Reddit said that Voxcommando has great voice recognition(even 
better than Siri from Apple). You can estimate it's quality here - 
http://www.youtube.com/watch?v=ewCRJZKFW34

Voxcommando runs on Windows OS only but you can use Virtualbox(just google it 
if you need it).

http://voxcommando.com/mediawiki/index.php?title=EventGhost this could probably 
help in integrating it into OpenHab.

I can't try this idea yet myself but hope it helps someone who wants to try it 
out.

Original comment by nfedyas...@gmail.com on 10 Nov 2013 at 1:53

[GoogleCodeExporter]

I don't think anything that is platform specific should be used. Also, 
Voxcommando is a commersial product and costs $30.

Original comment by mishoboss on 11 Nov 2013 at 9:20

[GoogleCodeExporter]

This issue has been migrated to Github. If this issue id is greater than103 its 
id has been preserved on Github. You can open your issue by calling the URL 
https://github.com/openhab/openhab/issues/<issueid>. Issues with ids less or 
equal 103 new ids were created.

Original comment by teichsta on 17 Nov 2013 at 8:08

• Added labels: Migrated-To-Github

[GoogleCodeExporter]

see above!

Issue has been migrated to Github and should be discussed there.

Original comment by teichsta on 21 Nov 2013 at 1:51

• Changed state: Invalid