chengsokdara / use-whisper

React hook for OpenAI Whisper with speech recorder, real-time transcription, and silence removal built-in
MIT License
732 stars 137 forks source link

Transcript object always undefined #7

Open ajesuscode opened 1 year ago

ajesuscode commented 1 year ago

When running a first code snippet provided in Readme I always get transcript text and blob as undefined Can you help or advise please?

chengsokdara commented 1 year ago

Hello @ajesuscode can you paste here the code you use? and may some logs that can help me identify the problem. I test on my end it is working. Make sure you provide correct OpenAI API token key.

ajesuscode commented 1 year ago

Hello again Here is a code snippet `import React, { useState } from "react"; import "./index.css"; import { useWhisper } from "@chengsokdara/use-whisper"; import { Configuration, OpenAIApi } from "openai";

// const configuration = new Configuration({ // apiKey: import.meta.env.OPENAI_API_KEY, // });

// const openai = new OpenAIApi(configuration); // const resp = await openai.createTranscription( // fs.createReadStream("audio.mp3"), // "whisper-1" // );

function App() { const { recording, speaking, transcribing, transcript, pauseRecording, startRecording, stopRecording, } = useWhisper({ apiKey: import.meta.env.VITE_OPENAI_API, // YOUR_OPEN_AI_TOKEN removeSilence: true, }); console.log(transcript); console.log(transcribing); console.log(speaking); return (

{transcript.text}

); }

export default App; ` Here is a logs

Screenshot 2023-03-11 at 23 36 07
chengsokdara commented 1 year ago

@ajesuscode after testing your snippet.

since you use removeSilence flag, transcript will return undefined blob and text if you don't speak. first test without speaking it is like you said, but when I speak ffmpeg-wasm fired up and start remove silence from your recording and spit out mp3.

try speaking and see if text is present, if not let me know the browser you test on.

below is my log with your snippet it works as expected.

test2.tsx?5174:29 {blob: undefined, text: undefined}
test2.tsx?5174:30 false
test2.tsx?5174:31 false
RecordRTC.js?8ae0:53 RecordRTC version:  5.6.2
RecordRTC.js?8ae0:63 started recording audio stream.
RecordRTC.js?8ae0:1057 Using recorderType: MediaStreamRecorder
RecordRTC.js?8ae0:2107 Passing following config over MediaRecorder API. {mimeType: 'audio/webm', type: 'audio', checkForInactiveTracks: false, recorderType: ƒ, initCallback: ƒ}
RecordRTC.js?8ae0:713 Recorder state changed: recording
RecordRTC.js?8ae0:103 Initialized recorderType: MediaStreamRecorder for output-type: audio
test2.tsx?5174:29 {blob: undefined, text: undefined}
test2.tsx?5174:30 false
test2.tsx?5174:31 false
RecordRTC.js?8ae0:129 Stopped recording audio stream.
RecordRTC.js?8ae0:713 Recorder state changed: stopped
RecordRTC.js?8ae0:170 audio/webm;codecs=opus -> 27.1 KB
test2.tsx?5174:29 {blob: undefined, text: undefined}
test2.tsx?5174:30 true
test2.tsx?5174:31 false
createFFmpeg.js?49ca:43 [info] use ffmpeg.wasm v0.11.6
createFFmpeg.js?49ca:43 [info] load ffmpeg-core
createFFmpeg.js?49ca:43 [info] loading ffmpeg-core
createFFmpeg.js?49ca:43 [info] ffmpeg-core loaded
createFFmpeg.js?49ca:43 [info] run FS.writeFile speech.webm <27056 bytes binary file>
createFFmpeg.js?49ca:43 [info] run ffmpeg command: -i speech.webm -acodec libmp3lame -aq 9 -ar 16000 -af silenceremove=start_periods=1:stop_periods=-1:start_threshold=-30dB:stop_threshold=-30dB:start_silence=2:stop_silence=2 speech.mp3
createFFmpeg.js?49ca:43 [fferr] ffmpeg version 3ea4213 Copyright (c) 2000-2020 the FFmpeg developers
createFFmpeg.js?49ca:43 [fferr]   built with emcc (Emscripten gcc/clang-like replacement) 2.0.8 (d059fd603d0b45b584f634dc2365bc9e9a6ec1dd)
createFFmpeg.js?49ca:43 [fferr]   configuration: --target-os=none --arch=x86_32 --enable-cross-compile --disable-x86asm --disable-inline-asm --disable-stripping --disable-programs --disable-doc --disable-debug --disable-runtime-cpudetect --disable-autodetect --extra-cflags='-O3 --closure 1 -I/src/build/include' --extra-cxxflags='-O3 --closure 1 -I/src/build/include' --extra-ldflags='-O3 --closure 1 -I/src/build/include -L/src/build/lib' --pkg-config-flags=--static --nm=llvm-nm --ar=emar --ranlib=emranlib --cc=emcc --cxx=em++ --objcc=emcc --dep-cc=emcc --disable-pthreads --disable-w32threads --disable-os2threads --enable-gpl --enable-nonfree --enable-zlib --enable-libx264 --enable-libx265 --enable-libvpx --enable-libwavpack --enable-libmp3lame --enable-libfdk-aac --enable-libtheora --enable-libvorbis --enable-libfreetype --enable-libopus --enable-libwebp --enable-libass --enable-libfribidi
createFFmpeg.js?49ca:43 [fferr]   libavutil      56. 51.100 / 56. 51.100
createFFmpeg.js?49ca:43 [fferr]   libavcodec     58. 91.100 / 58. 91.100
createFFmpeg.js?49ca:43 [fferr]   libavformat    58. 45.100 / 58. 45.100
createFFmpeg.js?49ca:43 [fferr]   libavdevice    58. 10.100 / 58. 10.100
createFFmpeg.js?49ca:43 [fferr]   libavfilter     7. 85.100 /  7. 85.100
createFFmpeg.js?49ca:43 [fferr]   libswscale      5.  7.100 /  5.  7.100
createFFmpeg.js?49ca:43 [fferr]   libswresample   3.  7.100 /  3.  7.100
createFFmpeg.js?49ca:43 [fferr]   libpostproc    55.  7.100 / 55.  7.100
createFFmpeg.js?49ca:43 [fferr] [opus @ 0x1564ac0] Warning: not compiled with thread support, using thread emulation
createFFmpeg.js?49ca:43 [fferr] Input #0, matroska,webm, from 'speech.webm':
createFFmpeg.js?49ca:43 [fferr]   Metadata:
createFFmpeg.js?49ca:43 [fferr]     encoder         : Chrome
createFFmpeg.js?49ca:43 [fferr]   Duration: N/A, start: 0.000000, bitrate: N/A
createFFmpeg.js?49ca:43 [fferr]     Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
createFFmpeg.js?49ca:43 [fferr] [opus @ 0x15655c0] Warning: not compiled with thread support, using thread emulation
createFFmpeg.js?49ca:43 [fferr] Stream mapping:
createFFmpeg.js?49ca:43 [fferr]   Stream #0:0 -> #0:0 (opus (native) -> mp3 (libmp3lame))
createFFmpeg.js?49ca:43 [fferr] [libmp3lame @ 0x1567d00] Warning: not compiled with thread support, using thread emulation
createFFmpeg.js?49ca:43 [fferr] Output #0, mp3, to 'speech.mp3':
createFFmpeg.js?49ca:43 [fferr]   Metadata:
createFFmpeg.js?49ca:43 [fferr]     TSSE            : Lavf58.45.100
createFFmpeg.js?49ca:43 [fferr]     Stream #0:0(eng): Audio: mp3 (libmp3lame), 16000 Hz, mono, fltp (default)
createFFmpeg.js?49ca:43 [fferr]     Metadata:
createFFmpeg.js?49ca:43 [fferr]       encoder         : Lavc58.91.100 libmp3lame
createFFmpeg.js?49ca:43 [fferr] size=       2kB time=00:00:00.72 bitrate=  23.6kbits/s speed=37.3x    
createFFmpeg.js?49ca:43 [fferr] video:0kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 11.792453%
createFFmpeg.js?49ca:43 [ffout] FFMPEG_END
createFFmpeg.js?49ca:43 [info] run FS.readFile speech.mp3
createFFmpeg.js?49ca:43 [Program terminated with exit(1)] undefined
RecordRTC.js?8ae0:771 RecordRTC is destroyed.
test2.tsx?5174:29 {blob: Blob, text: 'Hello.'}blob: Blob {size: 2133, type: 'audio/mpeg'}text: "Hello."[[Prototype]]: Object
test2.tsx?5174:30 false
test2.tsx?5174:31 false
ajesuscode commented 1 year ago

Thanx I had issue as ffmpeg was not installed. My fault. Thanx again!

dsantoro commented 10 months ago

I'm having the same problem but using Vite. I created exactly in both "platforms" Vite and RCA and I in Vite I have the same error as ajesuscode had. In RCA version it shows a lot of warning about ffmpeg but works fine.

tamselvan89 commented 8 months ago

I'm facing this issue

Transcript undefined.

RecordRTC is destroyed.

Pls help on this ASAP