Stream to HA Assist Pipeline

[bendikrb]

I'm a bit lost -- I have installed the rhasspy3 hassio addon, which isn't quite usable yet as far as I can tell.. And if it was, I still wouldn't get to where I want, which is this;

1. Do wake on rpi satellite
2. Stream audio to home assistant server (I want to do stt using a cloud service or local wyoming whisper server).
3. Stream TTS audio back to the satellite.

This could be accomplished by doing what the existing remote/websocket does, except against HA`s WebSocket API.

I have to assume I'm kicking in open doors here - but after reading around both here and there, I couldn't find any mention about this..

[bendikrb]

Never mind - I just discovered https://github.com/synesthesiam/homeassistant-pipeline/blob/master/audio_to_audio.py Thanks :)

[bendikrb]

.. it just needs a little tweak to read audio like the websocket program does, not from stdin..

[WarLL3]

may you explain how you realised this setup?

[benbender]

@bendikrb would you mind to share your process and config? Would be gorgeous! <3

[lightsabata]

@bendikrb i'm also interested how you manage to make it work

[bendikrb]

I didn't, actually. However, I'm encouraged to give it another try now that I see the latest commits to the wyoming-v1 branch + the pending pull requests from @Shulyaka

[benbender]

@bendikrb would love to see a working implementation!

[lightsabata]

with your advice i was able to make it work with the following method:

Download https://github.com/rhasspy/rhasspy3/tree/wyoming-v1

After configuring with the tutorial with your needs, you must install the program/remote/home_assistant

mkdir -p config/programs/remote/
cp -R programs/remote/home_assistant config/programs/remote/
config/programs/remote/home_assistant/script/setup

the edit your configuration (here's mine):

programs:
  mic:
    arecord:
      command: |
        arecord -q -r 16000 -c 1 -f S16_LE -t raw -
      adapter: |
        mic_adapter_raw.py --rate 16000 --width 2 --channels 1
  wake:
    <YOUR_PREFERED_WAKE_WORD_PROGRAM:
  snd:
    aplay:
      command: |
        aplay --device rhasspy -q -r 22050 -f S16_LE -c 1 -t raw
      adapter: |
        snd_adapter_raw.py --rate 22050 --width 2 --channels 1
  remote:
    home_assistant:
      command: |
        bin/assist_pipeline.py --token "${token}" --server "${server}"
      template_args:
        token: "<YOUR_LIVE_LONG_ACCESS_TOKEN>"
        server: "<YOUR_HOME_ASSISTANT_URI:PORT>"

satellites:
  default:
    mic:
      name: arecord
    wake:
      name: <YOUR_PREFERED_WAKE_WORD_PROGRAM>
    remote:
      name: home_assistant
    snd:
      name: aplay

Make sure to have piper and whisper on home assistant correctly configure (settings must be the same betwen add-ons configuration and vocal assistant setting in home assistant.

then you can run the following command to start listening on the rhasspy3 folder:

script/run bin/satellite_run.py --debug --loop

there is must be other and nicer ways to make it work and i would love to improve this

[synesthesiam]

Sorry everyone for all of the hacks needed to get things running :/ I'm heavily focused on wake word stuff for Home Assistant at the moment, but I will return to get things cleaned up in Rhasspy soon.

[Shulyaka]

This is a very early developer preview, so don't worry about that!

[lightsabata]

i agree with Shulyaka. I've just testing with nabu casa assistant cloud. it is pretty rock solid and at this state it work better than any voice assisant that i know so far! (speed, accuracy, customization) you've definitely earn my subscription ! By the way is there a method to output a sound after a wake word triggering ? (perhaps i'm a little too impatient)

[Shulyaka]

By the way is there a method to output a sound after a wake word triggering ?

Try this:

pipelines:
  default:
    wake:
      name: porcupine1
      after:
        command: |
          aplay activation.wav

[lightsabata]

Thanks, unfortunately it doesn't seems to work on my config

[Shulyaka]

Replace activation.wav with the path to the actual file you want to play. Also, it's not a full pipeline, just the important parts.

[lightsabata]

yes, i've also try with the full command (same as snd program) without success:

satellites:
  default:
    mic:
      name: arecord
    wake:
      name: porcupine2
      after: 
        command: |
          aplay --device rhasspy -q -r 22050 -f S16_LE -c 1 -t raw "${data_dir}/wake_alfred.wav"
    remote:
      name: home_assistant
    snd:
      name: aplay

[Shulyaka]

Ah, it the satellites! I don't know if it is supported, never tried it yet.

[lightsabata]

Does it work on your side on pipeline mode ? I tried too but it doesn't seems to work either (it might be a specific problem to my configuration)

EDIT: nevermind, i have not see that you've made a commit for that and it work on pipeline but not on satellite To not mess with this thread, i've created a new one : (https://github.com/rhasspy/rhasspy3/issues/47)

[RobertLukan]

I followed the instructions here and it works. I only had issue with this:

File "/home/pi/rhasspy3-wyoming/rhasspy3/config/programs/remote/home_assistant/bin/assist_pipeline.py", line 14, in import aiohttp ModuleNotFoundError: No module named 'aiohttp'

I have manually installed aiohttp.

I hope this tree gets merged with the main as it is very cool functionality. I was trying to achieve the same with wyoming-satellite but I am having several issues with the wake word while here wake word runs very nicely.