Keeps timing out

[marcjl1]

Describe the bug I've tried multiple setup configurations and source repos but can't get past a timeout error at the Notion_voice_notes step.

Which cloud storage app are you using? (Google Drive, Dropbox, or OneDrive) I've tried with both Dropbox and Google Drive

Have you tried updating your workflow? Please follow the steps here, and ensure you've tested the latest version of the workflow: https://thomasjfrank.com/how-to-transcribe-audio-to-text-with-chatgpt-and-notion/#update

I have updated to latest versions

Does the issue only happen while testing the workflow, or does it happen during normal, automated runs? Happens in both testing and normal runs.

Please paste the contents of your Logs tab from the notion_voice_notes action step. Exports

Inputs

Logs

Details 3/26/2024, 10:55:46 PM Checking that file is under 300mb...

3/26/2024, 10:55:46 PM File size is approximately 28.1mb.

3/26/2024, 10:55:46 PM File is under the size limit. Continuing...

3/26/2024, 10:55:46 PM Checking if the user set languages...

3/26/2024, 10:55:46 PM No language set. Whisper will attempt to detect the language.

3/26/2024, 10:55:46 PM Logging settings...

3/26/2024, 10:55:46 PM Object {14} 3/26/2024, 10:55:46 PM Setup stage duration: 1.619996ms

3/26/2024, 10:55:46 PM Total duration so far: 1.619996ms (0.001619996 seconds)

3/26/2024, 10:55:46 PM File path of Google Drive file: /tmp/10-07-32.m4a

3/26/2024, 10:55:46 PM Successfully got duration: 3368 seconds

3/26/2024, 10:55:46 PM Download stage duration: 322.510814ms (0.322510814 seconds)

3/26/2024, 10:55:46 PM Total duration so far: 324.13081ms (0.32413081 seconds)

3/26/2024, 10:55:46 PM Using OpenAI's Whisper service for transcription.

3/26/2024, 10:55:46 PM Chunking file: /tmp/10-07-32.m4a

3/26/2024, 10:55:46 PM Full file size: 26.777830123901367mb. Chunk size: 24mb. Expected number of chunks: 2. Commencing chunking...

3/26/2024, 10:55:52 PM Spliting file into chunks with ffmpeg command: /pipedream/dist/code/cb04459f3dacf569fcf828602f2d384647f27fe82fe8b1e6740b4c60d045e426/node_modules/.pnpm/@ffmpeg-installer+ffmpeg@1.1.0/node_modules/@ffmpeg-installer/linux-x64/ffmpeg -i "/tmp/10-07-32.m4a" -f segment -segment_time 1685 -c copy -loglevel verbose "/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-%03d.m4a"

3/26/2024, 10:55:54 PM stderr: ffmpeg version N-47683-g0e8eb07980-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2018 the FFmpeg developers built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516 configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg libavutil 56. 24.101 / 56. 24.101 libavcodec 58. 42.100 / 58. 42.100 libavformat 58. 24.100 / 58. 24.100 libavdevice 58. 6.101 / 58. 6.101 libavfilter 7. 46.101 / 7. 46.101 libswscale 5. 4.100 / 5. 4.100 libswresample 3. 4.100 / 3. 4.100 libpostproc 55. 4.100 / 55. 4.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/10-07-32.m4a': Metadata: major_brand : M4A minor_version : 0 compatible_brands: M4A mp42isom creation_time : 2024-03-22T15:03:40.000000Z Duration: 00:56:08.17, start: 0.000000, bitrate: 66 kb/s Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 64 kb/s (default) Metadata: creation_time : 2024-03-22T15:03:40.000000Z [segment @ 0x55cc380] Selected stream id:0 type:audio [segment @ 0x55cc380] Opening '/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-000.m4a' for writing Output #0, segment, to '/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-%03d.m4a': Metadata: major_brand : M4A minor_version : 0 compatible_brands: M4A mp42isom encoder : Lavf58.24.100 Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 64 kb/s (default) Metadata: creation_time : 2024-03-22T15:03:40.000000Z Stream mapping: Stream #0:0 -> #0:0 (copy) Press [q] to stop, [?] for help [segment @ 0x55cc380] segment:'/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-000.m4a' starts with packet stream:0 pts:0 pts_time:0 frame:0 size=N/A time=00:17:17.26 bitrate=N/A speed=2.07e+03x [segment @ 0x55cc380] segment:'/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-000.m4a' count:0 ended [AVIOContext @ 0x56187c0] Statistics: 14 seeks, 65 writeouts [segment @ 0x55cc380] Opening '/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-001.m4a' for writing [segment @ 0x55cc380] segment:'/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-001.m4a' starts with packet stream:0 pts:80880640 pts_time:1685.01 frame:78985 size=N/A time=00:31:25.09 bitrate=N/A speed=1.89e+03x size=N/A time=00:46:23.21 bitrate=N/A speed=1.84e+03x No more output streams to write to, finishing. [segment @ 0x55cc380] segment:'/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-001.m4a' count:1 ended [AVIOContext @ 0x55cf9c0] Statistics: 14 seeks, 65 writeouts size=N/A time=00:56:08.14 bitrate=N/A speed=1.86e+03x video:0kB audio:26717kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown Input file #0 (/tmp/10-07-32.m4a): Input stream #0:0 (audio): 157883 packets read (27358634 bytes); Total: 157883 packets (27358634 bytes) demuxed Output file #0 (/tmp/chunks-2eFlJEB8AIMPw5TEACtkcVZrvcG/chunk-%03d.m4a): Output stream #0:0 (audio): 157883 packets muxed (27358634 bytes); Total: 157883 packets (27358634 bytes) muxed [AVIOContext @ 0x55cfb40] Statistics: 28085332 bytes read, 3 seeks

3/26/2024, 10:55:54 PM Created 2 chunks.

3/26/2024, 10:55:54 PM Chunks created successfully. Transcribing chunks: chunk-000.m4a,chunk-001.m4a

3/26/2024, 10:55:54 PM Transcribing file: chunk-000.m4a

3/26/2024, 10:55:54 PM Transcribing file: chunk-001.m4a

3/26/2024, 10:57:23 PM Encountered error from OpenAI: Connection error.

3/26/2024, 10:57:23 PM Status code: undefined

3/26/2024, 10:57:23 PM Error name: Error

3/26/2024, 10:57:23 PM Error headers: undefined

3/26/2024, 10:57:23 PM Encountered a recoverable error. Retrying...

3/26/2024, 10:57:23 PM Retrying transcription for chunk-001.m4a due to error: Error: Connection error.

3/26/2024, 10:57:24 PM Encountered error from OpenAI: Connection error.

3/26/2024, 10:57:24 PM Status code: undefined

3/26/2024, 10:57:24 PM Error name: Error

3/26/2024, 10:57:24 PM Error headers: undefined

3/26/2024, 10:57:24 PM Encountered a recoverable error. Retrying...

3/26/2024, 10:57:24 PM Retrying transcription for chunk-000.m4a due to error: Error: Connection error.

3/26/2024, 10:57:24 PM Transcribing file: chunk-001.m4a

3/26/2024, 10:57:26 PM Transcribing file: chunk-000.m4a

3/26/2024, 10:58:54 PM Encountered error from OpenAI: Connection error.

3/26/2024, 10:58:54 PM Status code: undefined

3/26/2024, 10:58:54 PM Error name: Error

3/26/2024, 10:58:54 PM Error headers: undefined

3/26/2024, 10:58:54 PM Encountered a recoverable error. Retrying...

3/26/2024, 10:58:54 PM Retrying transcription for chunk-001.m4a due to error: Error: Connection error.

3/26/2024, 10:58:56 PM Encountered error from OpenAI: Connection error.

3/26/2024, 10:58:56 PM Status code: undefined

3/26/2024, 10:58:56 PM Error name: Error

3/26/2024, 10:58:56 PM Error headers: undefined

3/26/2024, 10:58:56 PM Encountered a recoverable error. Retrying...

3/26/2024, 10:58:56 PM Retrying transcription for chunk-000.m4a due to error: Error: Connection error.

3/26/2024, 10:58:58 PM Transcribing file: chunk-001.m4a

3/26/2024, 10:58:59 PM Transcribing file: chunk-000.m4a

3/26/2024, 11:00:26 PM Encountered error from OpenAI: Connection error.

3/26/2024, 11:00:26 PM Status code: undefined

3/26/2024, 11:00:26 PM Error name: Error

3/26/2024, 11:00:26 PM Error headers: undefined

3/26/2024, 11:00:26 PM Encountered a recoverable error. Retrying...

3/26/2024, 11:00:26 PM Retrying transcription for chunk-001.m4a due to error: Error: Connection error.

3/26/2024, 11:00:29 PM Encountered error from OpenAI: Connection error.

3/26/2024, 11:00:29 PM Status code: undefined

3/26/2024, 11:00:29 PM Error name: Error

3/26/2024, 11:00:29 PM Error headers: undefined

3/26/2024, 11:00:29 PM Encountered a recoverable error. Retrying...

3/26/2024, 11:00:29 PM Retrying transcription for chunk-000.m4a due to error: Error: Connection error.

3/26/2024, 11:00:33 PM Transcribing file: chunk-001.m4a

3/26/2024, 11:00:34 PM Transcribing file: chunk-000.m4a

[TomFrankly]

@marcjl1 this looks like it was caused by OpenAI's service being overloaded. I can see "Connection Error" serveral times in the logs, which can happen when too many people are trying to send requests to their servers.

If you try again later, it should probably work! Unfortunately, "Connection Error" is all OpenAI returns when this happens; it doesn't send back any additional details.

[marcjl1]

Hi Tom,

It must some other cause, as I've tried several times a day for over a week now and I always get the same result. I've tried creating a new API key from OpenAI and same result. Could the connection be failing for any other reason you're aware of?

Marc

[TomFrankly]

@marcjl1 I tried creating a brand new instance of the workflow from my template today, and was able to test successfully.

Have you double-checked that you've added money directly to OpenAI at platform.openai.com (not ChatGPT Plus subscription)?

[marcjl1]

@TomFrankly , I'm using an API key from a funded account at OpenAI. I see usage costs there from some other API calls used in my local apps. I just installed a new instance of your workflow in Pipedream and, sadly, still getting the same time out result. So frustrating. I was using this workflow successfully for a few months and used it often. Was really helpful in helping me capture notes from my many meetings a day.

Any other ideas on how to troubleshoot?

[TomFrankly]

@marcjl1 totally get how frustrating this must be! This is the most common issue that gets brought up here in the repo, but I've never seen it myself and have never been able to replicate it, no matter what I do.

I just tried re-creating the workflow from my template again and successfully transcribed a file: https://www.loom.com/share/fec12dcb2f1f470aa44da72985f5b504?sid=62892b6d-a228-4ff1-9631-5cfec8bfcf85

You mentioned your key is being used in other applications – are all of these making requests to the Chat endpoint specifically?

The only thing I can think of here is that the specific Whisper (transcription) endpoint could get overloaded more quickly than the Chat endpoint – perhaps OpenAI just hasn't dedicated enough resources to the Whisper model, and maybe it can't handle as many requests.

Perhaps they also prioritize requests from older, more established accounts – that could explain why I've never hit the connection error myself. I've tried creating a new account for testing purposes, but they require phone validation and only let you have one account per phone number. So I just can't do it.

If this is indeed the issue, one thing you could try is setting your Transcription Service setting to Deepgram (making sure you've updated the notion_voice_notes action):

This will require signing up for another API key, but it might solve the issue. Deepgram's transcription models are also cheaper, and in the case of Nova-2, much faster as well. I've personally been testing Nova-2 recently.

[marcjl1]

@TomFrankly , OK, I'm back in business :) I switched over to using Deepgram for the transcriptions, and that fixed the timeout on that step. I think I also figured out how to setup the funding of the OpenAI account properly, which may have also been preventing the workflow from working. Thanks for the tips.