search

chengsokdara / use-whisper

React hook for OpenAI Whisper with speech recorder, real-time transcription, and silence removal built-in
MIT License
736 stars 139 forks source link

Issue chaining actions #27

Closed athrael-soju closed 1 year ago

athrael-soju commented 1 year ago

First off, thanks for your work. I find this very useful. The base application is working fine. I can use all my buttons and I can get the transcript ok.

But where I am facing issues is when I want to chain the actions. For example, I want to:

  1. Press Start to record and then press Stop.
  2. Then I want the voiced prompt to be transcribed into text and fed into my google speech library to get a response.
  3. I want that response Audio to be played

My functions:

  1. startRecording -> stopRecording will generate the transcript.
  2. sendMessage(): Takes the transcript and prompts chatGPT, then returns the text response.
  3. listenAudio: Takes the text response and uses google TTS to voice the response.

My issue is getting an undefined transrcribe.text after stopRecording ends, so I can't feed it into sendMessage. I've tried a few different approaches and got close, but not quite there yet.

So, now It's all a manual task. Start, Stop, Send Message, Listen to response.

Any clues to make better use of the API and get transrcribe on demand?

athrael-soju commented 1 year ago

I was able to resolve this and currently using your hook in my repo: https://github.com/athrael-soju/whisperChat Feel free to check it out. Thanks!