search

bastibe / python-soundfile

SoundFile is an audio library based on libsndfile, CFFI, and NumPy
BSD 3-Clause "New" or "Revised" License
712 stars 108 forks source link

Initial segfault when writing OGG VORBIS that is too long #233

Open scopatz opened 5 years ago

scopatz commented 5 years ago

I have found that writing OGG VORBIS files where data is of dtype float32 and the sample rate is 22050, will succeed when writing data arrays of length 1048576 (2^20), but will segfault if the array is of length 2097152 (2^21). Using faulthandler, Python 3.6, and pysoundfile v0.10.2, the traceback is below:

Fatal Python error: Segmentation fault

Current thread 0x00007f777ac0e740 (most recent call first):
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/soundfile.py", line 1321 in _cdata_io
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/soundfile.py", line 1310 in _array_io
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/soundfile.py", line 990 in write
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/soundfile.py", line 316 in write
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/umdone/sound.xsh", line 221 in save
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/umdone/commands/save.py", line 43 in main
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/umdone/commands/__init__.py", line 36 in dec
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 13654 in wait
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 13871 in iterraw
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 13983 in tee_stdout
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 14070 in _end
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 14062 in end
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 21527 in run_subproc
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 21561 in subproc_captured_hiddenobject
  File "/home/scopatz/open-source-directions/edit.xsh", line 15 in <module>
  File "/home/scopatz/.local/lib/python3.6/site-packages/xonsh/__amalgam__.py", line 22401 in exec
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/umdone/main.xsh", line 35 in run
  File "/home/scopatz/miniconda/lib/python3.6/site-packages/umdone/main.xsh", line 94 in main
  File "/home/scopatz/miniconda/bin/umdone", line 3 in <module>
Segmentation fault (core dumped)

From the above, using pdb, I inserted a break right before the segfault, and found that the func being called is the libsndfile function sf_writef_float <cdata 'int64_t(*)(struct SNDFILE_tag *, void *, int64_t)' 0x7fcc369a9830>

What is weirder is that if I first call the write function with a working value for frames (ie 1048576), and then call it with a segfaulting value, such as 57889754, it will correctly save the value that would have otherwise segfaulted. See the following.

> /home/scopatz/miniconda/lib/python3.6/site-packages/soundfile.py(1321)_cdata_io()
-> frames = func(self._file, data, frames)
(Pdb) func(self._file, data, 1048576)
1048576
(Pdb) frames
57889754
(Pdb) func(self._file, data, frames)
57889754

This second save seems to actually work. I am not sure if I really understand what is going on here. Let me know if you need any more information.

ironbar commented 5 years ago

I can confirm the same problem when trying to save ogg files with audios with 2**21 or more samples.

nartes commented 4 years ago

A problem does still exist on linux at least. 16PCM, 44100 32 works, 44100 64 has a crash inside libvorbis

(gdb) where
#0  0x00007f261220b16c in ?? () from /usr/lib/libvorbis.so.0
#1  0x00007f261220bf58 in vorbis_analysis_wrote () from /usr/lib/libvorbis.so.0
#2  0x00007f261249991f in ?? () from /usr/lib/libsndfile.so.1
#3  0x00007f2612499d87 in ?? () from /usr/lib/libsndfile.so.1
#4  0x00007f2612478dfd in sf_writef_short () from /usr/lib/libsndfile.so.1
#5  0x00007f261252fdec in ffi_call_unix64 ()
   from /home/nartes/Documents/current/freelance-project-12/tmp/env3/lib/python3.8/site-packages/.libs_cffi_backend/libffi-806b1a9d.so.6.0.4
#6  0x00007f261252ef55 in ffi_call ()
   from /home/nartes/Documents/current/freelance-project-12/tmp/env3/lib/python3.8/site-packages/.libs_cffi_backend/libffi-806b1a9d.so.6.0.4
#7  0x00007f26127519e6 in cdata_call (cd=0x7f260bd25c60, args=<optimized out>, kwds=<optimized out>)
    at c/_cffi_backend.c:3148
#8  0x00007f26218ffad2 in _PyObject_MakeTpCall () from /usr/lib/libpython3.8.so.1.0
#9  0x00007f26219bc2f4 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#10 0x00007f26219a67a6 in ?? () from /usr/lib/libpython3.8.so.1.0
#11 0x00007f26219bc3c8 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#12 0x00007f26219a67a6 in ?? () from /usr/lib/libpython3.8.so.1.0
#13 0x00007f26219bc3c8 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#14 0x00007f26219a67a6 in ?? () from /usr/lib/libpython3.8.so.1.0
#15 0x00007f26219bc3c8 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#16 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#17 0x00007f26219a624b in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#18 0x00007f26219b8a9c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
--Type <RET> for more, q to quit, c to continue without paging--
#19 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#20 0x00007f2621a2e3d3 in PyEval_EvalCode () from /usr/lib/libpython3.8.so.1.0
#21 0x00007f2621a34e7d in ?? () from /usr/lib/libpython3.8.so.1.0
#22 0x00007f2621905e37 in ?? () from /usr/lib/libpython3.8.so.1.0
#23 0x00007f26219b7c8c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#24 0x00007f262191277a in ?? () from /usr/lib/libpython3.8.so.1.0
#25 0x00007f26219bc779 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#26 0x00007f262191277a in ?? () from /usr/lib/libpython3.8.so.1.0
#27 0x00007f26219bc779 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#28 0x00007f262191277a in ?? () from /usr/lib/libpython3.8.so.1.0
#29 0x00007f26219068e8 in ?? () from /usr/lib/libpython3.8.so.1.0
#30 0x00007f26219b80ce in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#31 0x00007f26219a606d in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#32 0x00007f26219b7c8c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#33 0x00007f26219a606d in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#34 0x00007f26219b80ce in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#35 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#36 0x00007f26219a6892 in ?? () from /usr/lib/libpython3.8.so.1.0
#37 0x00007f26219b8a9c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#38 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#39 0x00007f26219a624b in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#40 0x00007f26219b80ce in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#41 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#42 0x00007f26219a624b in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#43 0x00007f26219b80ce in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#44 0x00007f26219a606d in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#45 0x00007f26219b80ce in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#46 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#47 0x00007f26219a624b in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#48 0x00007f26219a6c73 in ?? () from /usr/lib/libpython3.8.so.1.0
#49 0x00007f26218f7418 in PyObject_Call () from /usr/lib/libpython3.8.so.1.0
#50 0x00007f26219b9d03 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#51 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#52 0x00007f26219a624b in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0
#53 0x00007f26219b7c8c in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#54 0x00007f26219a4e3b in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#55 0x00007f2621a2e3d3 in PyEval_EvalCode () from /usr/lib/libpython3.8.so.1.0
#56 0x00007f2621a2e428 in ?? () from /usr/lib/libpython3.8.so.1.0
#57 0x00007f2621a32623 in ?? () from /usr/lib/libpython3.8.so.1.0
#58 0x00007f26218cd3e7 in PyRun_FileExFlags () from /usr/lib/libpython3.8.so.1.0
#59 0x00007f26218d7f4a in PyRun_SimpleFileExFlags () from /usr/lib/libpython3.8.so.1.0
#60 0x00007f2621a3f8be in Py_RunMain () from /usr/lib/libpython3.8.so.1.0
#61 0x00007f2621a3f9a9 in Py_BytesMain () from /usr/lib/libpython3.8.so.1.0
#62 0x00007f2621baf023 in __libc_start_main () from /usr/lib/libc.so.6
#63 0x00005604e862c05e in _start ()
snakers4 commented 4 years ago

Segmentation fault (core dumped)

We confirm the same issues with

For audios longer than 100 - 120 seconds Our sampling rate is typically 16,000

Falling back to sox to convert longer audios for now (we usually cut audio first, but this time we decided to compress first)

bastibe commented 4 years ago

As far as I know, this is a bug in libsndfile and/or vorbis/opus, and there is nothing we can do within soundfile to fix it.