search

jwdj / EasyABC

EasyABC
GNU General Public License v2.0
87 stars 36 forks source link

Segfaults and other failures using fluidsynth on OpenSuSE 15.5 (Python 3.6 & 3.10) #79

Closed revad closed 5 months ago

revad commented 10 months ago

I described in https://github.com/jwdj/EasyABC/issues/76 how I got EasyABC working in OpenSuSE Leap 15.5 with Python 3.6 (which is still the default python version.) That describes how I got rid of the fluidsynth-related segfault https://github.com/jwdj/EasyABC/issues/28 and subsequently discovered that it was not using fluidsynth at all. So I set out to get it to work with fluidsnyth.

I replaced fluidsyth.py with the version offered by @bomm here: https://github.com/jwdj/EasyABC/issues/63#issuecomment-1370203164 That file appears to be a version of fluidsyth.py from here https://github.com/nwhitehead/pyfluidsynth plus class 'Player' from EasyABC's fluidsyth.py

FluidSynth runtime version is 2.3.0 python3-wxPython v4.1.1

That does try to play with fluidsynth. It prints: fluidsynth: Using PulseAudio driver fluidsynth: warning: Failed to set thread to high priority fluidsynth: warning: Failed to set thread to high priority

By tracing I see it detected /usr/share/sounds/sf2/FluidR3_GM.sf2

If I press Play I normally get this immediately:

malloc_consolidate(): invalid chunk size
Fatal Python error: Aborted

Thread 0x00007f54657fa700 (most recent call first):
  File "/usr/lib64/python3.6/threading.py", line 295 in wait
  File "/usr/lib64/python3.6/queue.py", line 164 in get
  File "easy_abc.py", line 1603 in run
  File "/usr/lib64/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/usr/lib64/python3.6/threading.py", line 884 in _bootstrap

Current thread 0x00007f55152eb740 (most recent call first):
  File "/usr/lib64/python3.6/site-packages/wx/core.py", line 2237 in MainLoop
  File "easy_abc.py", line 8808 in <module>
Aborted (core dumped)

On a couple of occasions it played the first note before failing.

I tried Python 3.10 (which is the alternative version available in the SuSE repos) in a venv. (wxPython-4.2.1) Having fixed enough of the problems to get it to run in 3.10 (already noted by @bomm) I got the same malloc_consolidate(): invalid chunk size error.

I note that the problem concerns queue.py threading.py. I suspect this error is not caused by fluidsynth .

I'm currently stuck on this.

revad commented 10 months ago

In Python 3.10 I accidentally discovered that if, after starting up, I display the file settings dialogue, then press play, it plays the whole tune. Subsequently it will fail with the one of a number of errors depending what I do with the cursor:

malloc_consolidate(): invalid chunk size
Fatal Python error: Aborted

Thread 0x00007f3c427fc700 (most recent call first):
  File "/usr/lib64/python3.10/threading.py", line 320 in wait
  File "/usr/lib64/python3.10/queue.py", line 171 in get
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/easy_abc.py", line 1603 in run
  File "/usr/lib64/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib64/python3.10/threading.py", line 973 in _bootstrap

Current thread 0x00007f3cb542d740 (most recent call first):
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/venv/lib64/python3.10/site-packages/wx/core.py", line 2262 in MainLoop
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/easy_abc.py", line 8808 in <module>

Extension modules: wx._core, wx._html, wx._stc, pygame.base, pygame.constants, pygame.rect, pygame.rwobject, pygame.surflock, pygame.bufferproxy, pygame.math, pygame.surface, pygame.display, pygame.draw, pygame.event, pygame.imageext, pygame.image, pygame.joystick, pygame.key, pygame.mouse, pygame.time, pygame.mask, pygame.pixelcopy, pygame.transform, pygame.font, pygame.mixer_music, pygame.mixer, pygame.scrap, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pygame.pypm (total: 41)
Aborted (core dumped)
corrupted size vs. prev_size
Fatal Python error: Aborted

Thread 0x00007fc10bfff700 (most recent call first):
  File "/usr/lib64/python3.10/threading.py", line 320 in wait
  File "/usr/lib64/python3.10/queue.py", line 171 in get
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/easy_abc.py", line 1603 in run
  File "/usr/lib64/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib64/python3.10/threading.py", line 973 in _bootstrap

Current thread 0x00007fc1842fe740 (most recent call first):
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/music_score_panel.py", line 95 in OnLeftButtonDown
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/venv/lib64/python3.10/site-packages/wx/core.py", line 2262 in MainLoop
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/easy_abc.py", line 8808 in <module>
python3: cairo-surface.c:930: cairo_surface_reference: Assertion `CAIRO_REFERENCE_COUNT_HAS_REFERENCE (&surface->ref_count)' failed.
Fatal Python error: Aborted

Thread 0x00007f2a96ffd700 (most recent call first):
  File "/usr/lib64/python3.10/threading.py", line 320 in wait
  File "/usr/lib64/python3.10/queue.py", line 171 in get
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/easy_abc.py", line 1603 in run
  File "/usr/lib64/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib64/python3.10/threading.py", line 973 in _bootstrap

Current thread 0x00007f2b0a124740 (most recent call first):
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/venv/lib64/python3.10/site-packages/wx/core.py", line 2262 in MainLoop
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/easy_abc.py", line 8808 in <module>
corrupted double-linked list
Fatal Python error: Aborted

Thread 0x00007f879effd700 (most recent call first):
  File "/usr/lib64/python3.10/threading.py", line 320 in wait
  File "/usr/lib64/python3.10/queue.py", line 171 in get
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_230812/EasyABC_fluidsynth/easy_abc.py", line 1603 in run
  File "/usr/lib64/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/usr/lib64/python3.10/threading.py", line 973 in _bootstrap

So this is looks like a wx problem. In Python 3.10 I have wxPython-4.2.1 which was compiled by pip (legacy setup.py install)

To get the settings dialogue to display at all I made the changes here: https://github.com/wxWidgets/Phoenix/issues/2038#issuecomment-1371077645

In Python 3.6 accessing the settings dialogue makes no difference. I just get the invalid chunk size That is using python3-wxPython v4.1.1

revad commented 10 months ago

'Invalid chunk size' probably refers to 'chunks' within the soundfont - it was using fluid-soundfont-gm.sf2. The module fluidsynth/src/sfloader/fluid_sffile.c refers to these chunks. Replacing it with a smaller sf2 file gets rid of that particular failure message. Instead it usually now segfaults, or fails with 'corrupted double-linked list'

Fluidsynth is capable of playing the whole tune once if I go through this odd sequence:

  1. Display the ABC file settings, and dismiss the dialogue
  2. Select some notes on a staff
  3. Unselect them with a left click
  4. Press play

After (or during) play it then fails with one of a number of messages depending on what you click. Sometimes the active thread is wx, sometimes it's fluidsynth. Behaviour is essentially the same whether Python 3.6 or 3.10

I suspect the problem is in fluidsynth because it works fine with Timidity. I'll change the title of the issue.

revad commented 8 months ago

End of Year summary of issue status ;) Using xfce on opensuse 15.5. Updated fluidsynth to 2.3.4 (https://download.opensuse.org/repositories/home:/ecsos:/ardour/15.5/) Python 3.6.15: wx version 4.1.1 Python 3.10.13: wx version 4.2.1 I'm using fluidsyth.py from here: https://github.com/jwdj/EasyABC/issues/63#issuecomment-1370203164 Fluidsynth uses the PulseAudio driver Also various other minor changes to get it to run in python 3.10. But the failures are the same in Python 3.6 and 3.10.

I get a variety of unpredictable failures, many (I suspect all) of which seem to be related to memory management: -Segmentation fault -malloc_consolidate(): invalid chunk size -free(): invalid pointer -corrupted double-linked list -double free or corruption (!prev) -corrupted size vs. prev_size

It will reliably fail by just pressing play - the failure depending on how soon I press it. Sometimes the score window remains blank. Occasionally it will start to play then fail.

I can usually play a tune by first opening the ABC Settings window, clicking a field, and closing it. Pressing play then plays the complete tune and follows the score. After playing it takes 10+ seconds for the pause to revert to play after which it either spontaneously fails, or fails if I press play again.

The error and traceback don't give any clue about what part of fluidsynth (or wx?) has actually failed - see examples above.

Happy new year!

markblinkhorn commented 5 months ago

@revad I have made some modifications to fluidsynth.py, mainly to correct a problem affecting it running on 64-bit Windows, but it might just help with the 'Unpredictable Behaviours' that you have mentioned. The modified version is in the repo at https://github.com/jwdj/EasyABC/blob/master/fluidsynth.py

Also, I'd be grateful for your feedback as I have only been able to test the mods on 64bit Windows and Linux bookworm - it would be nice to know whether it also works on OpenSuse.

Mark

revad commented 5 months ago

Thanks, Mark. That looks promising.

I took a new snapshot of master and set up a python 10 venv. (Python 3.10.13) I pip-installed wxpython (4.2.1), pyparsing (3.1.2), pygame (2.5.2) (all were cached so same versions as before)

[Note for SUSE users. If you have to build wxpython as I did you need to apply a workround for this bug: https://github.com/wxWidgets/Phoenix/issues/2422 You have to symlink lib64 to lib in pip's tmp directory during the build:

cd /tmp
rm -r pip-*                     to clear old pip build dirs
while running the build:
ls -al pip-install*             to find name of the directory xxxx
cd pip-install-xxxx
ls                              to find name of the directory suffix yyyy
cd wxpython_yyyy 
cd build/wxbld/gtk3
ln -s lib lib64

The build takes 10 minutes on my i7 machine so there's plenty of time! ]

Without any other changes I ran that. Edited output:

(venv) david@localhost:~/projects/python/easyABC/easyabc_git_1.3.8.7_240312/easyabc> python easy_abc.py
ALSA lib conf.c:4555:(snd_config_update_r) Cannot access file /usr/local/share/alsa/alsa.conf
ALSA lib seq.c:935:(snd_seq_open_noupdate) Unknown SEQ default
fluidsynth: Using PulseAudio driver
fluidsynth: warning: Failed to set thread to high priority

(easy_abc.py:2489): Gtk-CRITICAL **: 15:04:39.896: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(easy_abc.py:2489): Gtk-CRITICAL **: 15:04:39.896: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(easy_abc.py:2489): Gtk-CRITICAL **: 15:04:39.896: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar
 (several of those)

(Issue 1: Many of these:)
Traceback (most recent call last):
  File "/home/david/projects/python/easyABC/easyabc_git_1.3.8.7_240312/venv/lib64/python3.10/site-packages/wx/lib/rcsizer.py", line 153, in CalcMin
    size = wx.Size( reduce( operator.add, self.colWidths),
TypeError: Size(): arguments did not match any overloaded call:
  overload 1: too many arguments
  overload 2: argument 1 has unexpected type 'float'
  overload 3: argument 1 has unexpected type 'float'

(Issue 2:) Some tabs in settings are OK: abcm2ps, musicXML these are screwed up and unreadable: abc2midi, voices, file settings, colors

Did a cold restart to clear previous settings settings

(venv) david@localhost:~/projects/python/easyABC/easyabc_git_1.3.8.7_240312/easyabc> python easy_abc.py

(Issue 3:)
The executable b'/usr/bin/gs' could not be found

ALSA lib conf.c:4555:(snd_config_update_r) Cannot access file /usr/local/share/alsa/alsa.conf
ALSA lib seq.c:935:(snd_seq_open_noupdate) Unknown SEQ default
fluidsynth: Using PulseAudio driver
fluidsynth: warning: Failed to set thread to high priority

15:18:57: Debug: ScreenToClient cannot work when toplevel window is not shown
15:18:57: Debug: ScreenToClient cannot work when toplevel window is not shown
15:18:57: Debug: ScreenToClient cannot work when toplevel window is not shown
15:18:57: Debug: ScreenToClient cannot work when toplevel window is not shown

(easy_abc.py:3217): Gtk-WARNING **: 15:18:57.600: Theme directory /apps of theme adwaita-xfce has no size field

I've seen all those messages and fixed those issues previously.

I can load an ABC and play it. pavucontrol says it's using fluidsynth.

Over the next day or so I re-apply fixes for those issues and any others and document them all in a new issue.

Many thanks.