search

sepinf-inc / IPED

IPED Digital Forensic Tool. It is an open source software that can be used to process and analyze digital evidence, often seized at crime scenes by law enforcement or in a corporate investigation by private examiners.
Other
968 stars 219 forks source link

Investigate LBC->WAV conversion failures in our internal transcription cluster #1800

Open lfcnassif opened 1 year ago

lfcnassif commented 1 year ago

See https://github.com/sepinf-inc/IPED/pull/1799#issuecomment-1666498770

lfcnassif commented 1 year ago

Hi @hauck-jvsh, since this seems related to the transcription container environment, could you take a look at this?

hauck-jvsh commented 1 year ago

Sure, I will take a look on this.

hauck-jvsh commented 1 year ago

Could you share the samples with me?

lfcnassif commented 1 year ago

Thanks! I just have this one: https://github.com/sepinf-inc/IPED/pull/1799#issuecomment-1666497454

Maybe @tc-wleite can share others.

hauck-jvsh commented 1 year ago

Thank you, I didn't see it. I think for now it is enough, maybe if I'm able to fix for this file some other would be nice to test.

wladimirleite commented 1 year ago

@hauck-jvsh, I will send you privately a few other LBC audios I collected here.

wladimirleite commented 1 year ago

@hauck-jvsh, I will send you privately a few other LBC audios I collected here.

@hauck-jvsh, I just sent you 50 LBC audios through Teams.

hauck-jvsh commented 1 year ago

I ran some tests here and strangly, the mplayer throws the following error

Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders libavcodec version 58.134.100 (external) Cannot find codec 'libilbc' in libavcodec... ADecoder init failed :( ADecoder init failed :( Cannot find codec for audio format 0x63626C69. Audio: no sound Video: no video

However if I run the ffmpeg directly, ffmpeg -i sample.lbc teste.wav, it is able to convert the file. Ps, I had to separately install ffmpeg, so maybe the mplayer is coming with an incomplete ffmepg lib.

hauck-jvsh commented 1 year ago

Do you it is worth using ffmeg as fallback in the transcription cluster?

wladimirleite commented 1 year ago

Do you it is worth using ffmeg as fallback in the transcription cluster?

Getting (and maybe compiling?) the latest version of MPlayer didn't help?

hauck-jvsh commented 1 year ago

I didn't try to compile it, I just try to install the latest pack available for ubuntu 22.04. I can try to compile and see if it solves.

wladimirleite commented 1 year ago

I didn't try to compile it, I just try to install the latest pack available for ubuntu 22.04. I can try to compile and see if it solves.

I see... Well, that was just a guess, I am not a Linux user. I will ask @aberenguel to test on his environment.

aberenguel commented 1 year ago

I've just tried in Ubuntu 22.04 and 23.04. Not supported on either. Command mplayer -benchmark -vo null -vc null -srate 16000 -af format=s16le,channels=1 -ao pcm:fast:file=sample.wav sample.lbc

Ubuntu 22.04

MPlayer 1.4 (Debian), built with gcc-11 (C) 2000-2019 MPlayer Team
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing sample.lbc.
libavformat version 58.76.100 (external)
libavformat file format detected.
[ilbc @ 0x7fe660e6f8e0]Estimating duration from bitrate, this may be inaccurate
[lavf] stream 0: audio (ilbc), -aid 0
Load subtitles in ./
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
libavcodec version 58.134.100 (external)
Cannot find codec 'libilbc' in libavcodec...
ADecoder init failed :(
ADecoder init failed :(
Cannot find codec for audio format 0x63626C69.
Audio: no sound
Video: no video

BENCHMARKs: VC:   0.000s VO:   0.000s A:   0.000s Sys:-1766.428s = -1766.428s

Exiting... (End of file)

Ubuntu 23.04

MPlayer UNKNOWN-12 (C) 2000-2023 MPlayer Team
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing sample.lbc.
libavformat version 59.27.100 (external)
libavformat file format detected.
[ilbc @ 0x7f23d9e8ad00]Estimating duration from bitrate, this may be inaccurate
[lavf] stream 0: audio (ilbc), -aid 0
Load subtitles in ./
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
libavcodec version 59.37.100 (external)
Cannot find codec 'libilbc' in libavcodec...
ADecoder init failed :(
ADecoder init failed :(
Cannot find codec for audio format 0x63626C69.
Audio: no sound
Video: no video

BENCHMARKs: VC:   0.000s VO:   0.000s A:   0.000s Sys:-2082.586s = -2082.586s

Exiting... (End of file)
aberenguel commented 1 year ago

I ran some tests here and strangly, the mplayer throws the following error

Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders libavcodec version 58.134.100 (external) Cannot find codec 'libilbc' in libavcodec... ADecoder init failed :( ADecoder init failed :( Cannot find codec for audio format 0x63626C69. Audio: no sound Video: no video

However if I run the ffmpeg directly, ffmpeg -i sample.lbc teste.wav, it is able to convert the file. Ps, I had to separately install ffmpeg, so maybe the mplayer is coming with an incomplete ffmepg lib.

ffmpeg worked in both Ubuntu 22.03 (ffmpeg version 4.4.2-0ubuntu0.22.04.1) and 23.04 (ffmpeg version 5.1.2-3ubuntu1). Why not use ffmpeg instead of mplayer?

wladimirleite commented 1 year ago

Why not use ffmpeg instead of mplayer?

Thanks for testing, @aberenguel! As MPlayer is already used by IPED, I think it would be better to avoid another dependency. However, if MPlayer is not opening LBC audios on Linux (it works on Windows, which is odd), I think it is fine to use FFmpeg. Probably using it as a fallback, as @hauck-jvsh suggested, seems a better option to minimize possible side effects (e.g. the opposite situation, audios supported by MPlayer but not FFmpeg).

aberenguel commented 1 year ago

I tried in many OS using docker (Alpine, Debian, Fedora) and it has not worked in none of them. I also tried using smplayer snap package. Interesting is that smplayer plays the file sample.lbc, but the mplayer not.

$ sudo snap install smplayer
$ snap run --shell smplayer
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

$ mplayer sample.lbc
MPlayer 1.5-9 (C) 2000-2022 MPlayer Team
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing sample.lbc.
libavformat version 59.17.102 (internal)
libavformat file format detected.
[ilbc @ 0x560bc2739640]Estimating duration from bitrate, this may be inaccurate
[lavf] stream 0: audio (ilbc), -aid 0
Load subtitles in ./
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
libavcodec version 59.21.100 (internal)
Cannot find codec 'libilbc' in libavcodec...
ADecoder init failed :(
ADecoder init failed :(
Cannot find codec for audio format 0x63626C69.
Audio: no sound
Video: no video

Exiting... (End of file)
lfcnassif commented 1 year ago

As MPlayer is already used by IPED, I think it would be better to avoid another dependency.

I agree, this seems the best approach, if possible of course, since this seems an environment issue, not an IPED one...

Maybe this issue is related to how mplayer was built in official packages? Perhaps may exist a linking option so libavcodec would link to libilbc...

If building from source doesn't work and such linking option doesn't exist, I'm not against using ffmpeg as fallback as @hauck-jvsh suggested and if it is found in PATH.

lfcnassif commented 8 months ago

@hauck-jvsh could you check if this is still happening after we upgraded our cluster infrastructure to use proxmox VMs? Was the SO changed?

hauck-jvsh commented 8 months ago

I can check, but probably it still occur, we install the same Ubuntu in the VM's and we are using the same containers.