Simple flow assistance needed

[infinitymakerspace]

Hi there

I have been able to get to a point where node-red sees my instance of voice2json installed via deb package on raspberry pi.

Unfortunately the documentation is a bit vague as there are no simple flow examples to work from.

I have added words to the vocab file and sentences.ini and is transcribed perfectly from cli. But I need a flow to run on boot to be triggered by a sentence to trigger a relay for 20 seconds. I can already manually trigger the relay part of the flow but voice is still not running like it should

Urgently need assistance

Regards

[johanneskropf]

There are examples for every node in the examples category. They are described in the readme sections for every single node. If you import them from the examples section of the the nodered import menu every example flow comes with a comment node that describes its usage. There is also this part in the wiki that’s is still work in progress. Can you describe your setup to me? Do you want to start listening for a wake-word from the start of nodered and when this wake word is spoken start recording a command to be transcribed? Please describe in more detail what you want to achieve. If you already wrote your sentences to the sentences.ini from outside nodered the first step will be to make a new voice2json configuration from any of the nodes and enter the path to your language profile folder. Than click the load sentences button so that you can see and edit those sentences from within nodered. I guess you already trained the profile. For further steps you will have to give me more details on your setup and what exactly you want to achieve. Johannes edit Ps you can also find me over on the nodered or Rhasspy forum as JGKK

[infinitymakerspace]

Raspberry Pi 3b + Rasbian lite debian package of voice2json installed usb audio device is set as default and only and works via voice2json cli I need to trigger a flow that handles a relay via gpio. That flow currently works in manual execute voice2json input needs to trigger that flow. attatched is relay trigger flow

[infinitymakerspace]

If I need to use a wake word thats fine actually preferable and then request with intent and then trigger the relay flow

On Wed, 15 Jul 2020, 17:56 Johannes Kropf, notifications@github.com wrote:

There are examples for every node in the examples category. They are described in the readme sections for every single node. If you import them from the examples section of the the nodered import menu every example flow comes with a comment node that describes its usage. There is also this part https://github.com/johanneskropf/node-red-contrib-voice2json/wiki/Getting-started---Step-by-step-tutorial in the wiki that’s is still work in progress. Can you describe your setup to me? Do you want to start listening for a wake-word from the start of nodered and when this wake word is spoken start recording a command to be transcribed? Please describe in more detail what you want to achieve. If you already wrote your sentences to the sentences.ini from outside nodered the first step will be to make a new voice2json configuration from any of the nodes and enter the path to your language profile folder. Than click the load sentences button so that you can see and edit those sentences from within nodered. I guess you already trained the profile. For further steps you will have to give me more details on your setup and what exactly you want to achieve. Johannes edit Ps you can also find me over on the nodered or Rhasspy forum as JGKK

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/johanneskropf/node-red-contrib-voice2json/issues/3#issuecomment-658839332, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOR3576OWNQBRC7OBIIGHSTR3XGSDANCNFSM4O2UN3CQ .

[johanneskropf]

Lets start simple. So you will need a microphone node like either node-red-contrib-contrib-sox-utils or the mic node to get the audio stream from your mic into nodered. Lets start without a wakeword and with a manually triggered recording.

[{"id":"8949ab64.5f43e8","type":"tab","label":"Flow 3","disabled":false,"info":""},{"id":"de8c0877.6c7f8","type":"inject","z":"8949ab64.5f43e8","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"start","payloadType":"str","x":210,"y":220,"wires":[["302c2a0.672d856"]]},{"id":"302c2a0.672d856","type":"sox-record","z":"8949ab64.5f43e8","name":"","buttonStart":"msg","inputs":1,"inputSource":"1,0","byteOrder":"-L","encoding":"signed-integer","channels":1,"rate":16000,"bits":16,"gain":"0","lowpass":8000,"showDuration":false,"durationType":"forever","durationLength":0,"silenceDetection":"nothing","silenceDuration":"2.0","silenceThreshold":"2.0","outputFormat":"stream","manualPath":"","debugOutput":false,"x":370,"y":220,"wires":[["e5c055d7.ba08c"],[]]},{"id":"e5c055d7.ba08c","type":"voice2json-record-command","z":"8949ab64.5f43e8","name":"","voice2JsonConfig":"d282ff11.6aa1e8","inputField":"payload","outputField":"payload","x":610,"y":220,"wires":[["379a4a1f.625106","cd96691a.f23098"]]},{"id":"379a4a1f.625106","type":"change","z":"8949ab64.5f43e8","name":"stop","rules":[{"t":"set","p":"payload","pt":"msg","to":"stop","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":450,"y":280,"wires":[["302c2a0.672d856"]]},{"id":"cd96691a.f23098","type":"voice2json-stt","z":"8949ab64.5f43e8","name":"","voice2JsonConfig":"d282ff11.6aa1e8","inputField":"payload","controlField":"control","outputField":"payload","autoStart":true,"x":860,"y":220,"wires":[["c1bcc853.9ec218"]]},{"id":"c1bcc853.9ec218","type":"debug","z":"8949ab64.5f43e8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1050,"y":220,"wires":[]},{"id":"d282ff11.6aa1e8","type":"voice2json-config","z":"","profilePath":"/home/pi/de_kaldi-zamia","name":"","sentences":"[GetTime]\nwie spät ist es\nsag mir die uhrzeit\n\n[GetTemperature]\nwie ist die temperatur\nwie (heiß | kalt) ist es\n\n[GetGarageState]\nist das garagentor (offen | geschlossen)\n\n[ChangeLightState]\nlight_name = (wohnzimmerlampe | garagenlicht) {name}\nlight_state = (ein | aus) {state}\nschalte (die | das) <light_name> <light_state>","slots":[{"fileName":"rhasspy/number","managedBy":"external","fileContent":null,"executable":true}],"removeSlots":false}]

This needs sox-utils nodes and sox installed. When you imported that flow choose your microphone in the record node and change the config to your voice2json config in the voice2json node. Now if you inject start it will start recording your command from the microphone stream and than send the recorded command to the stt node where it will be transcribed and than output an object that include the transcription. Get back to me if you have questions. Once you have this working get back to me and we will work from there.

[infinitymakerspace]

Hi there, many thanks, will get to it right now :-) and test

[johanneskropf]

The steps to add a profile folder that already has sentences is as follows (just to make sure):

• Open any of the voice2json nodes
• click on the config dropdown and click add new
• in the config screen that opens put in the absolute path to the location of the language profile folder you want to use
• click the load sentences button in the top left of the sentences edit window and confirm to load existing sentences
• give a unique name to the config
• save config and set the voice2json nodes you want to use to this saved config
• if the profile is not trained yet set a train node to the config and inject a msg.payload of train
• the functions for each node are described in the info tab on the right side

[infinitymakerspace]

Training and voice2json palette nodes are already installed.

On Wed, 15 Jul 2020, 18:29 Johannes Kropf, notifications@github.com wrote:

The steps to add a profile folder that already has sentences is as follows (just to make sure):

• Open any of the voice2json nodes
• click on the config dropdown and click add new
• in the config screen that opens put in the absolute path to the location of the language profile folder you want to use
• click the load sentences button in the top left of the sentences edit window and confirm to load existing sentences
• give a unique name to the config
• save config and set the voice2json nodes you want to use to this saved config
• if the profile is not trained yet set a train node to the config and inject a msg.payload of train
• the functions for each node are described in the info tab on the right side

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/johanneskropf/node-red-contrib-voice2json/issues/3#issuecomment-658865528, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOR357YK3EGKPCWREKAYB6DR3XKNDANCNFSM4O2UN3CQ .

[infinitymakerspace]

Node-Red zero text output in debug window, but from cli there is output

[johanneskropf]

Do step by step: 1st Have you checked that the sox record node is set up properly? connect a debug to the second output of the record node and check the checkbox for detailed debug output at the bottom of the node config. And show me the debug output. Did you select the right input device?

2nd step Do you see the recording from stream status below the record node?

3rd step Plug headphones into the pi. Connect a play node with the headphones as output device selected to the output of the record node and see if you can listen to your recording.

Edit the yellow warn messages above come from the voice2json stt node and look fine. Do you see any errors?

[johanneskropf]

You will have to inject start by clicking on the inject node everytime you want to start recording a command in the example i posted above. Just to make sure thats understood.

[infinitymakerspace]

Ok I picked that up so I tested with your flow and added extra output to the debug node and that worked. Then I copied it to a new flow, took out the stop node and triggered so it keeps listening. and this is the out put

But I think I do understand the importance of having a wake word as it then listens for the command withou voice2json going bonkers so the first one with the stop added also works so its to add a wake word and then link that intent or sentence to trigger my relay flow . Thank you for you help thus far, awaiting your reply

[johanneskropf]

Right now when you press start without the stop when it detected that you finished speaking the command it will start recording/listening for a command again. So if you don’t speak a second command it will time out after 30 seconds and throw an error and than try to record a command again. So if you would try to use it this way in a real world scenario it would send any speech everytime you finished speaking to the stt node. This would give you a lot of random transcriptions and false positives as the stt node would try to transcribe anything you say that wasn’t even meant for it and try fit it into the limited language model defined by your sentences.

This why you need some way to only trigger the recording of a command when you actually want to speak one. That’s why there is wake words like hey siri , alexa or ok google and so on in every assistant as they wouldn’t work otherwise. As they have no way to know what is a command or what is just random audio/conversation without a trigger that precedes the command. This doesn’t have to be a wakeword. This could also be a button connected to a raspberry pi’s gpios or a button in a dashboard.

Voice2json comes with the hey mycroft wake word preinstalled as it uses mycroft precise engine for wake word detection. But you can also train your own wake word for precise and define that in the profile.yml. But be aware that training your own precise model is quite an involved process.

So to summarize: The wake word acts as a gate so that only speech gets recorded for command transcription that is actually intended for the assistant/voice2json and actually includes a command.

[johanneskropf]

Here is the same flow with an added wake word (by default hey mycroft)

[{"id":"8949ab64.5f43e8","type":"tab","label":"Flow 3","disabled":false,"info":""},{"id":"de8c0877.6c7f8","type":"inject","z":"8949ab64.5f43e8","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"start","payloadType":"str","x":210,"y":220,"wires":[["302c2a0.672d856"]]},{"id":"302c2a0.672d856","type":"sox-record","z":"8949ab64.5f43e8","name":"","buttonStart":"msg","inputs":1,"inputSource":"1,0","byteOrder":"-L","encoding":"signed-integer","channels":1,"rate":16000,"bits":16,"gain":"0","lowpass":8000,"showDuration":false,"durationType":"forever","durationLength":0,"silenceDetection":"nothing","silenceDuration":"2.0","silenceThreshold":"2.0","outputFormat":"stream","manualPath":"","debugOutput":false,"x":370,"y":220,"wires":[["cf71d299.5ca778"],[]]},{"id":"e5c055d7.ba08c","type":"voice2json-record-command","z":"8949ab64.5f43e8","name":"","voice2JsonConfig":"d282ff11.6aa1e8","inputField":"payload","outputField":"payload","x":850,"y":220,"wires":[["379a4a1f.625106","cd96691a.f23098"]]},{"id":"379a4a1f.625106","type":"change","z":"8949ab64.5f43e8","name":"listen","rules":[{"t":"set","p":"control","pt":"msg","to":"listen","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":710,"y":280,"wires":[["cf71d299.5ca778"]]},{"id":"cd96691a.f23098","type":"voice2json-stt","z":"8949ab64.5f43e8","name":"","voice2JsonConfig":"d282ff11.6aa1e8","inputField":"payload","controlField":"control","outputField":"payload","autoStart":true,"x":1100,"y":220,"wires":[["c1bcc853.9ec218"]]},{"id":"c1bcc853.9ec218","type":"debug","z":"8949ab64.5f43e8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1290,"y":220,"wires":[]},{"id":"cf71d299.5ca778","type":"voice2json-wait-wake","z":"8949ab64.5f43e8","name":"","voice2JsonConfig":"d282ff11.6aa1e8","inputField":"payload","controlField":"control","outputField":"payload","nonContinousListen":true,"x":580,"y":220,"wires":[["66c5eb62.a0180c"],["e5c055d7.ba08c"]]},{"id":"66c5eb62.a0180c","type":"debug","z":"8949ab64.5f43e8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":790,"y":160,"wires":[]},{"id":"d282ff11.6aa1e8","type":"voice2json-config","z":"","profilePath":"/home/pi/de_kaldi-zamia","name":"","sentences":"[GetTime]\nwie spät ist es\nsag mir die uhrzeit\n\n[GetTemperature]\nwie ist die temperatur\nwie (heiß | kalt) ist es\n\n[GetGarageState]\nist das garagentor (offen | geschlossen)\n\n[ChangeLightState]\nlight_name = (wohnzimmerlampe | garagenlicht) {name}\nlight_state = (ein | aus) {state}\nschalte (die | das) <light_name> <light_state>","slots":[{"fileName":"rhasspy/number","managedBy":"external","fileContent":null,"executable":true}],"removeSlots":false}]

Of course do t forget to set the correct voice2json profile in all nodes and correctly configure the record node for your input device before deploying (so same as for the previous flow). Here once you have injected start once it will continue to listen for a wakeword forever and everytime it hears it it will start recording a command. Once it things the command was finished it starts listening for a wake word again.

[infinitymakerspace]

ok let me create another tab and run. Many thanks. Will get back in a few.

[johanneskropf]

I really recommend you also read this section and the linked parts in the voice2json documentation:

https://github.com/johanneskropf/node-red-contrib-voice2json#advanced-topics

And read the Nodes section where the working of every node in the suite is explained in detail:

https://github.com/johanneskropf/node-red-contrib-voice2json#nodes

[infinitymakerspace]

Well the wake word method works :-) - I would need to make the wakeword hello, as where this is being used its needs to be that simple

[infinitymakerspace]

Just a slight wrinkle still need to manually start the flow by clicking on the blue tab on the inject node

[johanneskropf]

The documentation on training a very basic wakeword model can be found here:

https://github.com/MycroftAI/mycroft-precise/wiki/Training-your-own-wake-word#how-to-train-your-own-wake-word

be aware that such a short phrase as hello will give you lots of false positives.

To auto start just set the start inject node to inject once after 10 seconds or something from the inject nodes config.

[infinitymakerspace]

Ah ok, I will look to for another wake word, many thanks for the assist. Let me make some backups first and then experiment as this is very close to working :-)

[infinitymakerspace]

I managed to get it all working many thanks [ { "id": "80cde23.fdcfb2", "type": "tab", "label": "wakeword", "disabled": false, "info": "" }, { "id": "38e0c57c.a4530a", "type": "inject", "z": "80cde23.fdcfb2", "name": "", "props": [ { "p": "payload" } ], "repeat": "", "crontab": "", "once": false, "onceDelay": "5", "topic": "", "payload": "start", "payloadType": "str", "x": 110, "y": 400, "wires": [ [ "78fa719a.b761a" ] ] }, { "id": "78fa719a.b761a", "type": "sox-record", "z": "80cde23.fdcfb2", "name": "", "buttonStart": "msg", "inputs": 1, "inputSource": "0,0", "byteOrder": "-L", "encoding": "signed-integer", "channels": 1, "rate": 16000, "bits": 16, "gain": "0", "lowpass": 8000, "showDuration": false, "durationType": "forever", "durationLength": 0, "silenceDetection": "nothing", "silenceDuration": "2.0", "silenceThreshold": "2.0", "outputFormat": "stream", "manualPath": "", "debugOutput": false, "x": 270, "y": 140, "wires": [ [ "e8b85f5e.ed926" ], [] ] }, { "id": "82e25186.6ea89", "type": "voice2json-record-command", "z": "80cde23.fdcfb2", "name": "", "voice2JsonConfig": "aa3ded1f.a0f4d", "inputField": "payload", "outputField": "payload", "x": 730, "y": 240, "wires": [ [ "af12f0aa.66484", "210b3e1a.d60882" ] ] }, { "id": "af12f0aa.66484", "type": "change", "z": "80cde23.fdcfb2", "name": "listen", "rules": [ { "t": "set", "p": "control", "pt": "msg", "to": "listen", "tot": "str" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 570, "y": 480, "wires": [ [ "e8b85f5e.ed926" ] ] }, { "id": "210b3e1a.d60882", "type": "voice2json-stt", "z": "80cde23.fdcfb2", "name": "", "voice2JsonConfig": "aa3ded1f.a0f4d", "inputField": "payload", "controlField": "control", "outputField": "payload", "autoStart": true, "x": 940, "y": 180, "wires": [ [ "66d867d2.1d0458", "37a7238c.e9f1ac" ] ] }, { "id": "66d867d2.1d0458", "type": "debug", "z": "80cde23.fdcfb2", "name": "main", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "payload", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 1150, "y": 180, "wires": [] }, { "id": "e8b85f5e.ed926", "type": "voice2json-wait-wake", "z": "80cde23.fdcfb2", "name": "", "voice2JsonConfig": "aa3ded1f.a0f4d", "inputField": "payload", "controlField": "control", "outputField": "payload", "nonContinousListen": true, "x": 460, "y": 240, "wires": [ [], [ "82e25186.6ea89" ] ] }, { "id": "9f2c59d6.f72718", "type": "trigger", "z": "80cde23.fdcfb2", "name": "", "op1": "0", "op2": "1", "op1type": "num", "op2type": "num", "duration": "20", "extend": false, "units": "s", "reset": "", "bytopic": "topic", "topic": "topic", "outputs": 1, "x": 1290, "y": 300, "wires": [ [ "1bf8ffa2.e7934" ] ] }, { "id": "1bf8ffa2.e7934", "type": "rpi-gpio out", "z": "80cde23.fdcfb2", "name": "", "pin": "11", "set": true, "level": "1", "freq": "", "out": "out", "x": 1320, "y": 420, "wires": [] }, { "id": "37a7238c.e9f1ac", "type": "function", "z": "80cde23.fdcfb2", "name": "outputTrue", "func": "if (msg.payload.text == 'turn on the dispenser') {\nmsg.payload = 0\nreturn msg\n}", "outputs": 1, "noerr": 0, "initialize": "", "finalize": "", "x": 1060, "y": 380, "wires": [ [ "9f2c59d6.f72718", "f0f20a96.1a0d68" ] ] }, { "id": "f0f20a96.1a0d68", "type": "debug", "z": "80cde23.fdcfb2", "name": "FunctionDebugOut", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "payload", "targetType": "msg", "statusVal": "", "statusType": "auto", "x": 1300, "y": 600, "wires": [] }, { "id": "aa3ded1f.a0f4d", "type": "voice2json-config", "z": "", "profilePath": "/home/pi/.config/voice2json/", "name": "enUsKaldi", "sentences": "[GetTime]\nwhat time is it\ntell me the time\n\n[GetTemperature]\nwhats the temperature\nhow (hot | cold) is it\n\n[GetGarageState]\nis the garage door (open | closed)\n\n[ChangeLightState]\nlight_name = ((living room lamp | garage light) {name}) | \nlight_state = (on | off) {state}\n\nturn [the] \nturn [the] \n\n[ChangeLightColor]\nlight_name = (bedroom light) {name}\ncolor = (red | green | blue) {color}\n\nset [the] [to] \nmake [the] \n\n[ChangeDispenserState]\ndispenser_name = ((dispenser) {name})\ndispenser_state = (on | off) {state}\n\nturn [the] \nturn [the] \n", "slots": [ { "fileName": "rhasspy/number", "managedBy": "external", "fileContent": null, "executable": true } ], "removeSlots": false } ]