Add Snips Binding

[lolodomo]

I have the intention to develop a binding that would allow to use a part of Snips in openHAB. What I have in mind is to integrate Snips ASR, the Speak To Text feature of Snips, a still missing feature in openHAB (no binding included in the openHAB distribution to do that).

The binding will allow defining a thing for each Snips platform. It will handle hte connection to hte MQTT broker and will implement one Key Spotting service and one Speak-To-Text service that will then be usable in openHAB in combination with an openHAB Human Language Interpreter.

My idea is to use the Snips ASR in combination with the NLP (Natural Language Processor) that is part of openHAB Habot to get a good openHAB assistant to control the house (and without the requirement of Internet connection and without private data being analyzed by Amazon or Google).

[lolodomo]

For the users wanting to use Snips NLU, I could implement a channel triggered each time a Snips intent is detected. Then you could use this trigger channel in a rule to do what you want. The binding will in this case simplify stuff by hiding all the MQTT stuff.

[wborn]

I think you propose developing a new speech-to-text (STT) voice add-on instead of a binding?

Here's also a link to the related issue in OHC: https://github.com/openhab/openhab-core/issues/1063

[lolodomo]

My intention is a full binding, not just a STT binding. That means it will offer more than a STT service. Including a STT service in the binding would be a bonus but it is not possible with the current definition of a STT service in the core framework.

[lolodomo]

I was able to build a very first minimal version connecting to the Snips MQTT broker and getting messages from Snips topics. Playing just a little with Snips, I think Snips ASR is not a generic speak-to-text. It seems it only works in addition to installed Snips assistants (I have only the demo assistant installed). I think it is more or less written on their web site. If this is correct, it would make Snips ASR unusable as a generic STT. It looks like you can use the solution only if you build and install Snips assistants through their console/store.

[lolodomo]

One part of Snips code is open source, the Snips NLU. This looks to me like an alternative to OpenNLP currently used by HABot. We could have built an openHAB Human Language Interpreter with the Snips NLU library but unforutnately this library is in Python. I am not sure there is a simple way to call a Python library from Java.

[lolodomo]

My current intention is to get binding things linked to Snips intents. The binding will discover automatically these intents by listening to Snips platform (MQTT). The binding will implement few skills in charge to handle Snips intents, matching intent to OH items and running actions. For example, a skill to handle light on and another for light off. The user will be able to setup each thing (Snips intent) to select a skill and define Snips slot names where to find data. It will be possible to add new skills through additional OSGi bundles (similar to what is offered by HABot). Like HABot, this solution will require tags added to the items so that the binding can match the good items from an intent. I plan to use injection in ASR/NLU from openHAB to help detection (not yet tested).

Additionally, each thing will have a channel trigger that will provide the intent message as a JSON object for all the users that would prefer implementing their own handling code in an OH rule.

I also plan to publish an "official" Snips app containing predefined intents in French directly compatible with the binding without any setup.

I would be happy if I could get help to define these predefined intents we would like by default in openHAB and then to have someone implementing them in English (or other language) in Snips console (I will do it for French).

[lolodomo]

Very bad news: with Snips now bought by Sonos, this will be the end of Snips as open. They plan to close Snips console at the end of January. We have until this date to deploy a voice assistant from the console to our hardware. Then there will be no way to update/enhance the voice assistant. https://forum.snips.ai/t/important-message-regarding-the-snips-console/4145

By the way, as my work on openHAB side was almost finished, I will finish it and publish it. And I will try to publish something on Snips console before they close it, at least for French users.

But the Snips solution has unfortunately no future for us.

[kaikreuzer]

Yes, this is indeed very unfortunate that this neat solution disappears from the market. And it is indeed a very disrespectful move wrt the volunteer community that has been built up around Snips - the comments on the linked thread clearly show this.

It is another proof that unless something is fully open-source (and not merely "available for free"), it ain't future-proof.

[lolodomo]

I lose my time with this binding development!