search

beetbox / audioread

cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding for Python
MIT License
483 stars 108 forks source link

NoBackendError despite a backend (specifically FFmpeg) being installed #54

Closed uipo78 closed 1 year ago

uipo78 commented 7 years ago

Hello!

This is my first time posting an issue, so please cut me some slack if I'm not following some protocol!

I was trying to use python's librosa package on Windows 10 and encountered the following issue.

After running x, _ = librosa.load('data/fma_small/000/000002.mp3', sr = None) I receive this stack trace:

---------------------------------------------------------------------------
NoBackendError                            Traceback (most recent call last)
<ipython-input-2-15a8daa0e7fd> in <module>()
      1 start, end = 7, 17
      2 filename = utils.get_audio_path(AUDIO_DIR, 2)
----> 3 x, sr = librosa.load(filename, sr = None, mono = True)

Q:\Program Files\Anaconda3\lib\site-packages\librosa\core\audio.py in load(path, sr, mono, offset, duration, dtype, res_type)
    105 
    106     y = []
--> 107     with audioread.audio_open(os.path.realpath(path)) as input_file:
    108         sr_native = input_file.samplerate
    109         n_channels = input_file.channels

Q:\Program Files\Anaconda3\lib\site-packages\audioread\__init__.py in audio_open(path)
    112 
    113     # All backends failed!
--> 114     raise NoBackendError()

NoBackendError:

Out of curiosity, I also tried to run audioread.ffdec.FFmpegAudioFile('data/fma_small/000/000002.mp3') (since I know that FFmpeg is installed) but I received:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Q:\Program Files\Anaconda3\lib\site-packages\audioread\ffdec.py in __init__(self, filename, block_size)
    126                 stderr=subprocess.PIPE,
--> 127                 stdin=self.devnull,
    128             )

Q:\Program Files\Anaconda3\lib\site-packages\audioread\ffdec.py in popen_multiple(commands, command_args, *args, **kwargs)
     88         try:
---> 89             return subprocess.Popen(cmd, *args, **kwargs)
     90         except OSError:

Q:\Program Files\Anaconda3\lib\subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds)
    946                                 errread, errwrite,
--> 947                                 restore_signals, start_new_session)
    948         except:

Q:\Program Files\Anaconda3\lib\subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_start_new_session)
   1223                                          cwd,
-> 1224                                          startupinfo)
   1225             finally:

FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

NotInstalledError                         Traceback (most recent call last)
<ipython-input-6-be3b460211e8> in <module>()
----> 1 audioread.ffdec.FFmpegAudioFile(filename)

Q:\Program Files\Anaconda3\lib\site-packages\audioread\ffdec.py in __init__(self, filename, block_size)
    129 
    130         except OSError:
--> 131             raise NotInstalledError()
    132 
    133         finally:

NotInstalledError:

Again, I have FFmpeg installed, and ffmpeg works as a command in cmd.

Is anyone able to spot the problem?

Thanks!

sampsyo commented 7 years ago

Huh! That’s frustrating. Thank you for the detailed tracebacks. I’m guessing this has something to do with Windows executable search vagaries. Can you please try running this in a Python console?

>>> import subprocess
>>> subprocess.Popen(['ffmpeg'])

My best guess is that your cmd shell has the PATH variable extended to have ffmpeg available, but however your librosa installation somehow does not have the same PATH.

uipo78 commented 7 years ago

If I understand your guess correctly, then I think you're onto something: I ran what you asked, and subprocess.Popen(['ffmpeg']) returned exactly what running ffmpeg in cmd returns.

sampsyo commented 7 years ago

Got it. In that case, does it work to import audioread directly in the Python REPL and use the FFmpeg backend? If so, it may be necessary to examine the Windows environment settings on the program that’s using it indirectly.

carlthome commented 7 years ago

I get the exact same error (via librosa.load as well) but it seems to happen randomly when loading several files in succession.

It's hard to reproduce, but it happens frequently (every 50 files or so). This is all that I'm doing:

import librosa as lr

for audio_path in audio_paths:
    audio_samples = lr.load(audio_path, None)[0]

Could NoBackendError be more specific about what's actually breaking? Could it actually be obscuring an OutOfMemory or something?

sampsyo commented 7 years ago

No, we don't currently have more detail available in that exception. Perhaps you could try the same diagnosis steps as @uipo78 above—namely, directly using the backend you expect to exist, so you can see exactly where it's failing?

Also, does the exception happen reliably for a specific file? If so, it could be that whatever backend you're using fails to read that specific file.

andimarafioti commented 6 years ago

I'm having this same problem with any file with PyCharm. I debugged a bit and tried a few things:

The problem happens in line 89 of ffdec.py but inside the subprocess code. For my experiment in my system that line translates to: 

subprocess.Popen(['ffmpeg', '-i', "C:\\Users\\amarafioti\\PycharmProjects\\inpainting-similarity-graphs\\audio_files\\pop\\audio\\01-Sargon-Mindless.mp3", "-f", "s16le", "-"], stdin=devnull, stderr=-1, stdout=-1)

where -> devnull = open(os.devnull)

I changed the code in ffdec.py to:

    for i, command in enumerate(commands):
        cmd = [command] + command_args
        try:
            return subprocess.Popen(cmd, *args, **kwargs)
        except OSError as e:
            print(e)
            if i == len(commands) - 1:
                # No more commands to try.
                raise

and got this traceback: 

[WinError 2] The system cannot find the file specified

but stopping on the exception and doing os.path.isfile(path) returns True. Maybe that WinError refers to ffmpeg?

I then tried installing avconv as it is the other command that my audioread tries. It also failed to load it. I checked that both ffmpeg and avconv where installed both as user variables and system variables just in case but that didn't help.

I ended up just running my program in the console where it works perfectly. Did you also had this problem with PyCharm and/or an IDE?

sampsyo commented 6 years ago

That's strange! My best guess is that your PyCharm setup is controlling the PATH environment variable differently from the rest of your system (or the command line prompt, at least). That would mean that the ffmpeg or avconv command is not available for audioread to find it. Maybe that can be set explicitly in the IDE's project settings?

andimarafioti commented 6 years ago

YES, "fixed" it.

Following this intuition of the PATH environment variable behaving differently from the rest of my system I just copied ffmpeg to the parent folder of the directory and it worked! And then I realized I hardly ever close PyCharm, so I restarted it and it works! This issue also happens with terminals (environment variables don't take effect unless you open a new one).

@uipo78 maybe you had this same problem?

@carlthome it could be an OutOfMemory. Audioread just expects an OSError and there are a whole bunch of different exceptions that are OSErrors. You could debug that line or just change the code with what I proposed and see what gets printed.

carlthome commented 6 years ago

@carlthome it could be an OutOfMemory.

Yup, I think that was it actually. It would be nice if audioread gave clearer errors than NoBackendError.

andimarafioti commented 6 years ago

Can you still reproduce the error and debug to see if that was it? You should only set a breakpoint on the exception and remove avconv from the commands in line 32 of ffdec.py

sampsyo commented 6 years ago

Yeah, it would be interesting to explore a redesign of the error messages. The tricky thing is that we want to support automatic fallback between different backends, so if one backend can't be used, we don't want to immediately stop with a backend-specific error. But perhaps "aggregate" errors like NoBackendError could accumulate some context from each backend that failed?

TimSchmeier commented 6 years ago

I'm experiencing difficulty related to this and created this SO post:

https://stackoverflow.com/questions/46774309/zipping-conda-environment-breaks-audioreads-backend-python-pyspark

Any guidance would be appreciated. Thanks.

AnthonyTheKoala commented 6 years ago

I have solved my problem. The summary is to (a) close python and/or Idle, (b) run the appropriate installation packages, and (c) setting the environment variables for ffmpeg and appending the environment variable to the path at https://github.com/librosa/librosa/issues/743

Regards Anthony of Sydney NSW

pratistha commented 6 years ago

@carlthome I am facing the exactly same error and its random as you mentioned. Do you know how to solve this issue?

carlthome commented 6 years ago

Not really, no. I eventually started calling ffmpeg via subprocess.Popen and am pretty happy with more explicit control for my purposes.

import shlex
import subprocess

import numpy as np

def decoder(path, duration=3.0, channels=2, sample_rate=44100, dtype=np.float32):
    formats = {np.int16: 's16le', np.float32: 'f32le'}
    args = f'''
    ffmpeg 
    -v error
    -n
    -i "{path}" 
    -f {formats[dtype]} 
    -ac {channels} 
    -ar {sample_rate} 
    -
    '''
    args = shlex.split(args)

    buffer_size = int(channels * np.dtype(dtype).itemsize * duration * sample_rate)

    with subprocess.Popen(args, stdout=subprocess.PIPE) as pipe:
        while True:
            buffer = pipe.stdout.read(buffer_size)
            audio = np.frombuffer(buffer, dtype=dtype)
            audio = np.reshape(audio, (-1, channels))
            yield audio
            if len(buffer) < buffer_size:
                break

for x in decoder('piano.wav'):
    print(x.shape)
from IPython.display import Audio

audio = np.concatenate(list(decoder('piano.wav')))
Audio(audio.T, rate=44100)

If you're having trouble with audioread you could of course try libsoundfile and see if that works:

import soundfile as sf

y, sr = sf.read('audio.ogg')
anurag-as commented 6 years ago

Huh! That’s frustrating. Thank you for the detailed tracebacks. I’m guessing this has something to do with Windows executable search vagaries. Can you please try running this in a Python console?

>>> import subprocess
>>> subprocess.Popen(['ffmpeg'])

My best guess is that your cmd shell has the PATH variable extended to have ffmpeg available, but however your librosa installation somehow does not have the same PATH.

Can you please tell how to solve this issue? I'm currently running on Ubuntu 18LTS and python3.6

Interspatial commented 5 years ago

I'm running windows 10, installed ffmpeg 4.1 in my conda env, and I get a nobackenderror. Any help?

CorentinJ commented 5 years ago

You will get a NoBackend() exception if ffmpeg fails to read your file (e.g. if you passed a non-audio file by mistake). Double-check that you passed a valid path to an audio file. You can read the output of ffmpeg by putting a breakpoint or print(line) at line 195 of audioread/ffdec.py.

anurag-as commented 5 years ago

Hello!

This is my first time posting an issue, so please cut me some slack if I'm not following some protocol!

I was trying to use python's librosa package on Windows 10 and encountered the following issue.

After running x, _ = librosa.load('data/fma_small/000/000002.mp3', sr = None) I receive this stack trace:

---------------------------------------------------------------------------
NoBackendError                            Traceback (most recent call last)
<ipython-input-2-15a8daa0e7fd> in <module>()
      1 start, end = 7, 17
      2 filename = utils.get_audio_path(AUDIO_DIR, 2)
----> 3 x, sr = librosa.load(filename, sr = None, mono = True)

Q:\Program Files\Anaconda3\lib\site-packages\librosa\core\audio.py in load(path, sr, mono, offset, duration, dtype, res_type)
    105 
    106     y = []
--> 107     with audioread.audio_open(os.path.realpath(path)) as input_file:
    108         sr_native = input_file.samplerate
    109         n_channels = input_file.channels

Q:\Program Files\Anaconda3\lib\site-packages\audioread\__init__.py in audio_open(path)
    112 
    113     # All backends failed!
--> 114     raise NoBackendError()

NoBackendError:

Out of curiosity, I also tried to run audioread.ffdec.FFmpegAudioFile('data/fma_small/000/000002.mp3') (since I know that FFmpeg is installed) but I received:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Q:\Program Files\Anaconda3\lib\site-packages\audioread\ffdec.py in __init__(self, filename, block_size)
    126                 stderr=subprocess.PIPE,
--> 127                 stdin=self.devnull,
    128             )

Q:\Program Files\Anaconda3\lib\site-packages\audioread\ffdec.py in popen_multiple(commands, command_args, *args, **kwargs)
     88         try:
---> 89             return subprocess.Popen(cmd, *args, **kwargs)
     90         except OSError:

Q:\Program Files\Anaconda3\lib\subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds)
    946                                 errread, errwrite,
--> 947                                 restore_signals, start_new_session)
    948         except:

Q:\Program Files\Anaconda3\lib\subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_start_new_session)
   1223                                          cwd,
-> 1224                                          startupinfo)
   1225             finally:

FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

NotInstalledError                         Traceback (most recent call last)
<ipython-input-6-be3b460211e8> in <module>()
----> 1 audioread.ffdec.FFmpegAudioFile(filename)

Q:\Program Files\Anaconda3\lib\site-packages\audioread\ffdec.py in __init__(self, filename, block_size)
    129 
    130         except OSError:
--> 131             raise NotInstalledError()
    132 
    133         finally:

NotInstalledError:

Again, I have FFmpeg installed, and ffmpeg works as a command in cmd.

Is anyone able to spot the problem?

Thanks!

Hi, I don't know if it may solve your problem. But for me, some audio files gave this error. So I tried it with different audio files. Most worked, less than 1% of the audio files gave me this error from a huge data set in my project. So check it out. The problem may be due to a specific audio file.

kubicndmr commented 5 years ago

which properties of audio file cause this problem?

danmackinlay commented 5 years ago

Yes, I get this error if my audio fiels are corrupt or truncated. Note that I also get it sometimes if I load non-corrupt audio files in a multi-threaded context, for example in a pytorch batch data loader which AFAICT is not support, so that is also a possible way of getting this "catch all" error"

Christilut commented 5 years ago

I found out that you also get this error even if you have ffmpeg in your path (or in my case, next to your python executable) but with missing DLL's.

ffmpeg tries to run but will error due to missing DLL's and audioread interprets an erroring ffmpeg as "not available".

raffaelbd commented 4 years ago

YES, "fixed" it.

Following this intuition of the PATH environment variable behaving differently from the rest of my system I just copied ffmpeg to the parent folder of the directory and it worked! And then I realized I hardly ever close PyCharm, so I restarted it and it works! This issue also happens with terminals (environment variables don't take effect unless you open a new one). I m having same problems , despite installing ffmpeg. please give me a solution.

my code was: D = [] # Dataset for row in valid_data.itertuples(): y, sr = librosa.load('F:\paper\Bangla ASR\segmented/augmented 1/' + (row.path))
ps = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=52) if ps.shape != (52, 130): continue D.append( (ps, row.classID) )

NoBackendError Traceback (most recent call last)

in 1 D = [] # Dataset 2 for row in valid_data.itertuples(): ----> 3 y, sr = librosa.load('F:\paper\Bangla ASR\segmented/augmented 1/' + (row.path)) 4 ps = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=52) 5 if ps.shape != (52, 130): continue ~\.conda\envs\RaffaelEnv\lib\site-packages\librosa\core\audio.py in load(path, sr, mono, offset, duration, dtype, res_type) 117 118 y = [] --> 119 with audioread.audio_open(os.path.realpath(path)) as input_file: 120 sr_native = input_file.samplerate 121 n_channels = input_file.channels ~\.conda\envs\RaffaelEnv\lib\site-packages\audioread\__init__.py in audio_open(path) 114 115 # All backends failed! --> 116 raise NoBackendError() NoBackendError: N.B: I restarted jupyter notebook so many times but still problem exists
jaccqo commented 3 years ago

try installing ffmpeg with conda ,i was experiencing the same issue and worked for me after installing it with conda ,and not pip,i think its a path problem

detrin commented 2 years ago

Hello, I was also using librosa when the same error occurred when using ffmpeg binary manually added to PATH. Installing av with conda helped resolve the issue.

Anyway, it would be great if we could prevent future issues like these.

Steps to reproduce:

~/local_python_libs/audioread/ffdec.py in init(self, filename, block_size) 180 181 # Read relevant information from stderr. --> 182 self._get_info() 183 184 # Start a separate thread to read the rest of the data from

~/local_python_libs/audioread/ffdec.py in _get_info(self) 228 if not line: 229 # EOF and data not found. --> 230 raise CommunicationError("stream info not found") 231 232 # In Python 3, result of reading from stderr is bytes.

CommunicationError: stream info not found

- original command from librosa
```python
amplitudes, sampling_rate = librosa.load(path, sr=None, mono=False)

What is the issue?

Proposed solution? Check whether ffmpeg uses static linking if possible. Advice against installing ffmpeg from https://johnvansickle.com/ffmpeg/. Possibly ad github action that checks compatibility of ffmpeg with this library.

sampsyo commented 2 years ago

That's interesting! I still don't quite see what's wrong with that particular ffmpeg… is it broken altogether? Or is it just not reporting the stream information?

detrin commented 2 years ago

That's interesting! I still don't quite see what's wrong with that particular ffmpeg… is it broken altogether? Or is it just not reporting the stream information?

If I tried to edit locally audioread via Popen, the error is in function available() in ffdec.py. FFmpeg ends with return code 1. However, it runs via the command line. I am not quite sure why it doesn't run via Popen. However, I am pretty sure that the problem is that the binary I used is static build with statically linked libraries such as glibc . I use an older kernel (because I simply have to, it is not my machine, and I don't have root privileges). FFmpeg in conda package AV is built with dynamically linked libraries, so there is no such error with older version glibc.

Quick fix: It would be great to check in audioread for such a type of error and advice for installing ffmpeg from suggested sources or building it without statically linked libraries. I am sure many people run into similar issues, and new people will eventually visit this thread.

Longer fix: Perhaps setup of github actions with simple pytest tests would be reasonable so that in future errors of this nature are prevented. There are many projects using this repository https://github.com/beetbox/audioread/network/dependents, and there is currently many issues with NoBackendError() error https://github.com/search?q=NoBackendError%28%29+audioread&type=issues

I would be happy to elaborate and contribute.

sampsyo commented 2 years ago

Ah, got it. Thanks for clarifying. I think the thing to do would be to nail down exactly why running the tool on the command line works butPopening it does not. If we can understand that, then maybe we can either (a) fix it, or (b) detect those specific conditions and issue a better error message.

Maybe looking carefully at the stderr/stdout from the process before it exits with status 1 would help shed some light?

detrin commented 2 years ago

I will try to reproduce the error and create a small docker container with these steps.

detrin commented 2 years ago

Okay, so running the ffmpeg without error in cmd was possible due to later installation of ffmpeg with AV. When I try to run it now inside folder with "./" it doesn't run at all.

$ ./ffmpeg
Segmentation fault

Did I install the correct architecture? My architecture is

$   uname -a  
Linux XXX 5.4.17-2136.302.7.2.1.el7uek.x86_64 #2 SMP Tue Jan 18 13:44:44 PST 2022 x86_64 x86_64 x86_64 GNU/Linux

and I installed ffmpeg-git-amd64-static.tar.xz.

Again we can look at libc that I am using

$   ldd --version
ldd (GNU libc) 2.17
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

If we look into build info that John provided it says 


     Notes:  A limitation of statically linking glibc is the loss of DNS resolution. Installing
             nscd through your package manager will fix this.

John uses gcc 8.3.0 and looking into https://ftp.gnu.org/gnu/gcc/gcc-8.3.0/ we can see in NEWS

     * Support has been added for __builtin_cpu_is() and
       __builtin_cpu_supports(), allowing for very fast access to
       AT_PLATFORM, AT_HWCAP, and AT_HWCAP2 values. This requires use of
       glibc 2.23 or later.

but I am not sure whether this applies to amd64 architecture.

Processor that I am using

$   lscpu 
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                72
On-line CPU(s) list:   0-71
Thread(s) per core:    2
Core(s) per socket:    18
Socket(s):             2
NUMA node(s):          4
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz
Stepping:              7
CPU MHz:               2450.644
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000
BogoMIPS:              5200.00
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              25344K
NUMA node0 CPU(s):     0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,64,68
NUMA node1 CPU(s):     1,5,9,13,17,21,25,29,33,37,41,45,49,53,57,61,65,69
NUMA node2 CPU(s):     2,6,10,14,18,22,26,30,34,38,42,46,50,54,58,62,66,70
NUMA node3 CPU(s):     3,7,11,15,19,23,27,31,35,39,43,47,51,55,59,63,67,71
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke avx512_vnni md_clear flush_l1d arch_capabilities

so I think I downloaded the correct binary.

sampsyo commented 2 years ago

Aha—thanks for the digging! One way or another, the underlying cause seems to be a broken ffmpeg. I’m not entirely sure what our library should do differently, in that case—maybe the thing to do would be to expand the docs and suggest that, if you get this error and you have ffmpeg, you should try double-checking that your ffmpeg actually works?

detrin commented 2 years ago

maybe the thing to do would be to expand the docs and suggest that, if you get this error and you have ffmpeg, you should try double-checking that your ffmpeg actually works?

Yes, I totally agree. Maybe the package could also differentiate between Segmentation fault and OSError. With error message something FFmpegSegmentationFault: Please make sure your ffmpeg is running properly. For installation advice of ffmpeg please check . OSError: Please make sure that ffmpeg is installed on your system or that it is added to PATH. For installation advice of ffmpeg please check .

I see that this repository has Github Actions already, would it be possible to create daily github actions for checking the compatibility for a) compiled version of ffmpeg for stable version (and maybe latest) b) installation via apt c) installation via conda d) downloading from John's site and adding to PATH there could be four github actions for that running daily and badges could be added to the ffmpeg installation section in README. So, in case anyone gets one error message above, can quickly spot what is not working. Does it sound reasonable?

detrin commented 2 years ago

Also, why is there a check for avconv?

sampsyo commented 2 years ago

Any chance you might be interested in starting a PR to add some advice to the documentation?

Unfortunately, improving the error message is a little more complicated than just raising a different exception. For two reasons:

detrin commented 2 years ago

Any chance you might be interested in starting a PR to add some advice to the documentation?

Yes! But I am not sure how quickly I will finish the PR (master thesis and work, but I will find some time). It will be my first PR into a project widely used by the community, which is exciting. You can assign me the issue. I will test it in a newly created public repo in case of new GitHub actions.

A segmentation fault is a pretty generic type of problem to run into. We could guess when it happens based on the return code, but it's not actually clear that this information would be actionable in a specific way for most users. It might be simpler to have people who think they have FFmpeg just type ffmpeg on the command line and discover the root cause of the problem.

I agree, but we could provide that in recommendation inside NoBackendError specifically for ffdec.py.

We could raise a more specific exception, but that wouldn't surface automatically for most users. The library works, by design, by "falling back" to other backends to try reading the file another way. So the NoBackendError indicates that no backend succeeded. Immediately raising an error about FFmpeg's segfault, and aborting the fallback path, would prevent other backends from working when FFmpeg is broken.

We could gather more detailed info in available_backends() then pass it to audio_open(). For example table of backends with status info. If a user has ffmpeg, but it returns segfault, and other backend works, it makes sense to ignore that error and continue audio_open(). Perhaps adding the optional argument backend would also be reasonable.

detrin commented 2 years ago

Also, why is there a check for avconv?

Fork from FFmpeg

The Libav project was a fork of the project. It was announced on March 13, 2011 by a group of FFmpeg developers. The event was related to an issue in project management and different goals: FFmpeg supporters wanted to keep development velocity in favour of more features, while Libav supporters and developers wanted to improve the state of the code and take the time to design better APIs.

Confusion

At the beginning of this fork, Libav and FFmpeg separately developed their own versions of the ffmpeg command. Libav then renamed their ffmpeg to avconv to distance themselves from the FFmpeg project. During the transition period, when a Libav user typed ffmpeg, there was a message telling the user that the ffmpeg command was deprecated and avconv has to be used instead. This confused some users into thinking that FFmpeg (the project) was dead.

https://en.wikipedia.org/wiki/Libav

I see, but I have never heard of this fork.

detrin commented 2 years ago

bump

Could you assign me the issue please?

sampsyo commented 2 years ago

It would be great to nail down a concrete plan! I think the first step is to just amend the docs with some advice about broken ffmpeg installations?

detrin commented 2 years ago

It would be great to nail down a concrete plan! I think the first step is to just amend the docs with some advice about broken ffmpeg installations?

I am on it.

detrin commented 1 year ago

Some, time passed by and I am closing old issues on github. I wonder where are the docs to this package? Is it just README? @sampsyo

detrin commented 1 year ago

Okay, so in my company this issue was resolved by using ffmpeg from conda build and adding this build into PATH

ffmpeg version 5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10.3.0 (GCC)
  configuration: --prefix=/opt/cloudera/parcels/ANACONDA-2023-02-23_165004 --cc=/home/conda/feedstock_root/build_artifacts/ffmpeg_1649114005999/_build_env/bin/x86_64-conda-linux-gnu-cc --disable-doc --disable-openssl --enable-demuxer=dash --enable-gnutls --enable-gpl --enable-hardcoded-tables --enable-libfreetype --enable-libopenh264 --enable-vaapi --enable-libx264 --enable-libx265 --enable-libaom --enable-libsvtav1 --enable-libxml2 --enable-libvpx --enable-pic --enable-pthreads --enable-shared --disable-static --enable-version3 --enable-zlib --enable-libmp3lame --pkg-config=/home/conda/feedstock_root/build_artifacts/ffmpeg_1649114005999/_build_env/bin/pkg-config
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

https://anaconda.org/conda-forge/ffmpeg This is way how to install it locally without internet. Of course it would be possible to build the ffmpeg from source, but I can imagine that most of the users don't want to get into that.

Propose for the information would be

NoBackendError despite a backend (specifically FFmpeg) being installed

Make sure that you can execute ffmpeg properly. If you are getting segmentation fault

$ ffmpeg
Segmentation fault

Try installing ffmpeg with your OS package manager (apt, dnf, yum, etc.) or with conda. When you installed ffmpeg without package managers it may be possible that you have statically linked libraries which prevent you from using ffmpeg in the first place.

sampsyo commented 1 year ago

Hi there! Yes, the only documentation I have written in the README. Thanks for the note here about making sure FFmpeg is actually working; I hope it helps folks.

detrin commented 1 year ago

Hi there! Yes, the only documentation I have written in the README. Thanks for the note here about making sure FFmpeg is actually working; I hope it helps folks.

so if this looks alright to you I would make PR with added FAQ and "NoBackendError despite a backend (specifically FFmpeg) being installed" would be the first section in it. Would be that alright?

sampsyo commented 1 year ago

Sounds good. I might be interested in trimming things down to simplify, but a section like this seems like the right thing. Maybe it should be called "Troubleshooting" or similar?

detrin commented 1 year ago

Okay @sampsyo so here is new proposal then

Troubleshooting

NoBackendError despite a backend (specifically FFmpeg) being installed

Make sure that you can execute ffmpeg properly. If you are getting a segmentation fault. Try installing ffmpeg with your OS package manager (apt, dnf, yum, etc.) or with conda. Make then sure that ffmpeg is executable.

sampsyo commented 1 year ago

Sounds great!

detrin commented 1 year ago

Created https://github.com/beetbox/audioread/pull/135

detrin commented 1 year ago

@sampsyo I believe now the issue can be closed.