IoT assistant w/ LMs [NodeRED]

[mbz4]

IDEA: use NodeRED w/ ollama... and build an IoT assistant

minimally require 3 components in the flow:

something to handle speech to text (STT), another thing for text to speech (TTS) and a language model (LM) inference on the local gateway; a few possible packages listed here:

• Ollama node: https://flows.nodered.org/node/node-red-contrib-ollama

• STT node: https://www.npmjs.com/package/node-red-contrib-streaming-stt

• TTS node: https://flows.nodered.org/node/node-red-contrib-tts-ultimate

• OpenAI node: https://github.com/rozek/node-red-flow-openai-api

• OpenAI node: https://flows.nodered.org/node/@technithusiast/node-red-contrib-ai-intent

• chatGPT node: https://flows.nodered.org/node/node-red-contrib-custom-chatgpt

  • https://www.youtube.com/watch?v=97NyAxG8uY8

Need also:

• Mic input node
• speaker output node
• optional sensors, actuators for custom context feeds

[ulno]

Already did successful integration of festival for tts and rhasspy for stt (check ulno.net YouTube videos), maybe start from there?

For language recognition: https://rhasspy.readthedocs.io or http://voice2json.org/

For tts festival could be enough. Try Nina here: https://www.cstr.ed.ac.uk/projects/festival/morevoices.html