search

DFRobot / DFRobot_DF2301Q

MIT License
14 stars 11 forks source link

Access Raw Audio after a Wake Word #9

Open JpEncausse opened 5 months ago

JpEncausse commented 5 months ago

Following this discussion and my comment :

It would be very very valuable if we could have access to the raw audio after a wake word. For instance : "Hello bot, (... some audio ...)"

  1. Because it open the possibility to call LLM like ChatGPT and play back answer
  2. Because offline trigger to get the audio is very smart compared to always listening that send tons of request.
  3. Or at least a sample code to record audio just after a wakeword (assuming we do not loss some part) ?

Many thanks for your help. I can test the GPT parts but looking for a sample to get started retrieveing the audio from microphone.

qsjhyy commented 5 months ago

Unfortunately, this module does not support such functionality. Perhaps you can consider turning off the audio playback function of this module and adding a speech synthesis module to achieve this effect.

JpEncausse commented 5 months ago

@qsjhyy For the speech synthesis I simply want to play an audio downloaded fron the cloud. It is the easy part because many online tools provide that audio. I do not need speech synthesis on board.

My issue is related on speech recognition. I want to get the audio buffer from the device after voice trigger. To send it to the cloud and perform Speech2Text online. The API do not provide access to the microphone buffer ?

qsjhyy commented 5 months ago

The fact is, this module does not have the function that you say. This is more like the function of a standalone microphone.

JpEncausse commented 5 months ago

Ahhhh ok, I though a software workaround would be possible (since it's a microphone) but I understand it is a hardware limitation.

That would be awesome if DFRobot that kind of hardware. Here is an exemple with Grok, no not the Elon LLM, not the CPU, but a plush that embed a microphone connected to the cloud.

qsjhyy commented 5 months ago

DFRobot will release this kind of hardware in the future, so stay tuned.

JpEncausse commented 5 months ago

Thanks ! And I assume I also can't play raw audio using the speaker ? I have to add another Speaker and Microphone to the ESP-32 If I want to add dynamic question/answer