[Bug]: failed to transcode mp3 file

slimooo commented 4 months ago

What happened?

A bug happened!

Steps to reproduce

step one...
step two...

What OS are you seeing the problem on?

No response

Relevant log output

options: {
  "path": "/Users/slim/Desktop/13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3",
  "model_path": "/Users/slim/Library/Application Support/github.com.thewh1teagle.vibe/ggml-medium.bin",
  "lang": "fr",
  "verbose": false,
  "n_threads": 4,
  "init_prompt": "",
  "temperature": 0.4
}

Caused by:
    Invalid data found when processing input

Location:
    core/src/audio/encoder.rs:175:9
App Version: 1.0.7
Commit Hash: 99ae746dc02135ad7a27ec0f9adafe016b8c96e4
Arch: aarch64
Platform: macos
Kernel Version: 14.0.0
OS: macos
OS Version: 14.0.0
Models: ggml-medium.bin
Default Model: ggml-medium.bin"

thewh1teagle commented 4 months ago

@slimooo Does it work with other files? This file may be corrupt, can you check if the file is working when try to play it?

Thanks for reporting!

slimooo commented 4 months ago

Hello, i tried with this file : https://rf.proxycast.org/95275034-f537-415d-a3c7-8dc37213c669/13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3 It's doesn't work.

thewh1teagle commented 4 months ago

Not sure why it failed on your end. I tried to transcribe it in Windows and macOS and it worked (Transcribed to french)

Can you try to transcribe this file? short.wav.

slimooo commented 4 months ago

With this file, it works correctly. I have other files where it also works perfectly. In fact, it works perfectly with all files except with the podcasts from the French radio channel ‘France Inter’ https://www.radiofrance.fr/podcasts.

thewh1teagle commented 4 months ago

@slimooo

I tried to transcribe this specific file from the website: France Inter.mp4 It works perfectly on macOS :)

Maybe try it too. I also created new pre release version for Vibe for macOS vibe_1.0.9_coreml_metal_use_gpu_whisper_1.6.2.dmg Maybe it's some old bug which got fixed in latest versions of whisper.cpp

slimooo commented 4 months ago

I still encounter the problem :s

1.  I updated and did the test: bug
2.  I uninstalled and did the test: bug
3.  I moved the file: bug

… I don’t know what else to do!

Is your Mac running on Apple Silicon?

options: { "path": "/Users/slim/Downloads/13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3", "model_path": "/Users/slim/Library/Application Support/github.com.thewh1teagle.vibe/ggml-medium.bin", "lang": "fr", "verbose": false, "n_threads": 4, "init_prompt": "", "temperature": 0.4, "translate": null }

Caused by: Invalid data found when processing input

Location: core/src/audio/encoder.rs:175:9

thewh1teagle commented 4 months ago

@slimooo I'm running on apple silicon mac.

Can you try this specific file? 13939-27.02.2024-ITEMA_23660860-2024F29767S0058-27.m4a Looks like the one you tried is other but I need that we'll compare the same file so I can try reproduce the problem on my mac :)

oleole39 commented 4 months ago

Hello, i tried with this file : https://rf.proxycast.org/95275034-f537-415d-a3c7-8dc37213c669/13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3 It's doesn't work.

I tested that same file on Linux Mint 21 (Ubuntu 22.04-based with an Intel CPU in a non-Apple computer) with Vibe 2.0.1 (amd64 deb release) and immediately got the following same error at launching the transcription:

options: {
  "path": "/path/to/file/13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3",
  "model_path": "/path/to/models/github.com.thewh1teagle.vibe/ggml-medium.bin",
  "lang": "fr",
  "verbose": false,
  "n_threads": 4,
  "init_prompt": "",
  "temperature": 0.4,
  "translate": null
}

Caused by:
    Invalid data found when processing input

Location:
    core/src/audio/encoder.rs:175:9

Same result with lang set as auto or en.

Can you try to transcribe this file? short.wav.

Working.

Can you try this specific file? 13939-27.02.2024-ITEMA_23660860-2024F29767S0058-27.m4a

Working.

thewh1teagle commented 4 months ago

Hello, i tried with this file : https://rf.proxycast.org/95275034-f537-415d-a3c7-8dc37213c669/13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3 It's doesn't work.

This file worked for me on macOS, Windows, and Linux (ubuntu 22.04) Are you sure the audio file you downloaded is valid and playable?

oleole39 commented 4 months ago

This file worked for me on macOS, Windows, and Linux (ubuntu 22.04) Are you sure the audio file you downloaded is valid and playable?

Strange. I tried again with the very file you linked in that last message. Not sure what valid exactly means for you, but it is a file of 47707100 bytes of 49mn29s, with extension mp3, definitely playable with VLC for instance.

VLC indicates codec MPEG Audio Layer 1/2 (mpga).
Running ffprobe -select_streams a:0 -show_entries stream=codec_name -of default=noprint_wrappers=1:nokey=1 13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3 returns 'mp3':

```txt ffprobe version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2007-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared WARNING: library configuration mismatch avcodec configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libaribb24 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc --enable-libsmbclient libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 Input #0, mp3, from '13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3': Metadata: title : 1er février 2003, l’accident de la navette spatiale Columbia album : Affaires sensibles artist : Fabrice Drouelle, Franck COGNARD copyright : Radio France encoded_by : Radio France genre : Podcast date : 2024-05-20 master_object_id: NET_MFI_6929A934-2DAA-43AE-8B78-E60B0126CF76 lyrics : Aujourd’hui dans Affaires Sensibles, l’accident de la navette spatiale Columbia encoder : Lavf58.45.100 Duration: 00:49:27.85, start: 0.011995, bitrate: 128 kb/s Stream #0:0: Audio: mp3, 44100 Hz, stereo, fltp, 128 kb/s Metadata: encoder : Lavf Stream #0:1: Video: png, rgb24(pc), 400x400, 90k tbr, 90k tbn, 90k tbc (attached pic) Metadata: comment : Cover (front) mp3 ```

This command would work: ffmpeg -ss 00:00:03 -i 13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3 -to 00:00:08 -c:a pcm_s16le test_extract.wav And the resulting wav file is playable. So it seems ffmpeg version 4.4.2-0ubuntu0.22.04.1 can deal with the file, right?

However, when inputing that wav file to Vibe and launching the transcription, the following popup error is immediately fired:

options: {
  "path": "/path/to/file/test_extract.wav",
  "model_path": "/path/to/models/github.com.thewh1teagle.vibe/ggml-medium.bin",
  "lang": "fr",
  "verbose": false,
  "n_threads": 4,
  "init_prompt": "",
  "temperature": 0.4,
  "translate": null
}

Caused by:
    Argument invalide

Location:
    core/src/audio/encoder.rs:149:9

oleole39 commented 4 months ago

And actually:

ffmpeg -ss 00:00:03 -i 13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3 -to 00:00:08 -c:a libvorbis test_extract.ogg

brings a few errors in the conversion log, although the result file is playable:

[mp3float @ 0x5638964f86c0] Header missing
Error while decoding stream #0:0: Invalid data found when processing input
[mp3float @ 0x5638964f86c0] Header missing
Error while decoding stream #0:0: Invalid data found when processing input
[mp3float @ 0x5638964f86c0] Header missing
Error while decoding stream #0:0: Invalid data found when processing input
[ogg @ 0x5638964f3a80] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or -vsync 2

error with Vibe:

Caused by:
Opération non permise
Location:
core/src/audio/mod.rs:23:9

ffmpeg -ss 00:00:03 -i 13940-21.05.2024-ITEMA_23748018-2024F22805S0142-22.mp3 -to 00:00:08 -c:a aac test_extract.aac works.
- Working with Vibe! (no error pop-up at least, but transcription is only accurate if adding céan, à un an in the prompt box - I assume this is because the text extract is short and randomly cut).

thewh1teagle commented 3 months ago

I finally found some potential cause for this issue. Should be already fixed in 2.0.2

oleole39 commented 3 months ago

I finally found some potential cause for this issue. Should be already fixed in 2.0.2

Congrats, I confirm it is fixed on my side for all files tested in my 2 previous messages (original mp3, test wav and test ogg). Just being curious - briefly, did you get why it was working on your platform and not others?

thewh1teagle commented 3 months ago

did you get why it was working on your platform and not others?

I didn't found exactly why it worked on my end but it failed on your system. But I had serious bug there, previously I used ffmpeg libraries directly which are very hard to use compared to use ffmpeg binary. I found that ffmpeg authors recommend themself to use ffmpeg binary directly instead, and while replacing it, I found that I had issues in the encoder I wrote. Replacing the libaries with using ffmpeg directly solved it. I'm glad that the issue resolved!

thewh1teagle / vibe