search

ps0ares / CustomShaders

Custom Shaders is a LUA script for Open Broadcaster Software (OBS) that can be used to apply OBS effects to any image source.
17 stars 2 forks source link

OBS locks up sometimes when loading shaders #1

Open bldewolf opened 3 years ago

bldewolf commented 3 years ago

Hello,

I found your plugin while looking for lua examples and (as a fellow linux user) love the idea of a shader plugin that has no compiling requirements. While trying it out, though, I have been getting OBS to lock up when pressing the load shader button. It seems like a race condition somewhere, since it doesn't break on every press.

I also tried building a portable obs from git (27.0.1-4-g6942bb814) but it has the same issue so I don't think it's a conflict with other plugins that I have installed.

My general reproduction steps are this:

  1. add the script
  2. add a source
  3. add the filter to the source
  4. open the configuration dialog of the filter
  5. click load script repeatedly to reload the default shader
  6. eventually it locks up obs and has to be killed

Let me know if there's any other info that would be helpful to reproduce. I'm running OBS in XFCE on Archlinux.

ps0ares commented 3 years ago

Hello and thank you for your report.

I've been looking at these crashes but sadly I'm at a dead end. There's nothing related to the crashes in the log files and the crashes can happen after any number of reloads. I tried to isolate some part of the script code where the problem may be occurring but it turns out it can crash in several places that seem unrelated to me.

Any advice on how to further debug this problem is very welcome.

ogmkp commented 2 years ago

Hi, same problem here on obs 27.2.4, debian 11.

stephematician commented 1 year ago

Still seems to be an issue with OBS 29. In case this provides any hints (I'm sure you've managed to get further than I have) I ran OBS under GDB, once it crashed I interrupted the program ... here's the output of where:

(gdb) where
#0  futex_wait (private=0, expected=2, futex_word=0x555557667e60)
    at ../sysdeps/nptl/futex-internal.h:146
#1  __GI___lll_lock_wait (futex=futex@entry=0x555557667e60, private=0)
    at ./nptl/lowlevellock.c:49
#2  0x00007ffff3c980dd in lll_mutex_lock_optimized (mutex=0x555557667e60)
    at ./nptl/pthread_mutex_lock.c:48
#3  ___pthread_mutex_lock (mutex=0x555557667e60) at ./nptl/pthread_mutex_lock.c:128
#4  0x00007fffc3fae25e in  () at /usr/lib/x86_64-linux-gnu/libobs-scripting.so.1
#5  0x00007ffff6974be1 in obs_source_properties ()
    at /usr/lib/x86_64-linux-gnu/libobs.so.0
#6  0x0000555555659dce in  ()
#7  0x00007ffff456817e in QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const () at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#8  0x00007ffff456956a in QMetaObject::invokeMethod(QObject*, char const*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) () at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#9  0x0000555555681927 in  ()
#10 0x00007ffff697dacb in signal_handler_signal ()
    at /usr/lib/x86_64-linux-gnu/libobs.so.0
#11 0x00007ffff6974e9d in obs_source_update_properties ()
    at /usr/lib/x86_64-linux-gnu/libobs.so.0
#12 0x00007ffee937ba7d in  () at /usr/lib/x86_64-linux-gnu/obs-scripting/obslua.so
#13 0x00007fffb4031fb7 in  () at /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#14 0x00007fffb4042f13 in lua_pcall () at /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#15 0x00007fffc3faef0f in  () at /usr/lib/x86_64-linux-gnu/libobs-scripting.so.1
#16 0x000055555565ed24 in  ()
#17 0x00007ffff45ac273 in  () at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#18 0x00007ffff56adcb6 in QAbstractButton::clicked(bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#19 0x00007ffff56adf4e in  () at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#20 0x00007ffff56af694 in  () at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#21 0x00007ffff56af8c3 in QAbstractButton::mouseReleaseEvent(QMouseEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#22 0x00007ffff55d90e8 in QWidget::event(QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#23 0x00007ffff557fd36 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#24 0x00007ffff55892d7 in QApplication::notify(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#25 0x00007ffff4553a58 in QCoreApplication::notifyInternal2(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#26 0x00007ffff5587de2 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#27 0x00007ffff55eb80b in  () at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#28 0x00007ffff55ee635 in  () at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#29 0x00007ffff557fd36 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6
#30 0x00007ffff4553a58 in QCoreApplication::notifyInternal2(QObject*, QEvent*) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#31 0x00007ffff4bcb6bf in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) () at /usr/lib/x86_64-linux-gnu/libQt6Gui.so.6
#32 0x00007ffff4c13c8c in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt6Gui.so.6
#33 0x00007fffea9b386e in  ()
    at /usr/lib/x86_64-linux-gnu/qt6/plugins/platforms/../../../libQt6XcbQpa.so.6
#34 0x00007ffff311bd3b in g_main_context_dispatch () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#35 0x00007ffff31706c8 in  () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#36 0x00007ffff31193e3 in g_main_context_iteration () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#37 0x00007ffff477beae in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#38 0x00007ffff4560adb in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
    at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#39 0x00007ffff455c0f3 in QCoreApplication::exec() ()
    at /usr/lib/x86_64-linux-gnu/libQt6Core.so.6
#40 0x0000555555603a9e in main ()

Looks painful to debug - something in a call to obs_source_update_properties via OBS's lua. I doubt I can provide anything else without compiling OBS, and it looks like it's a problem on the OBS side (it's implementation of locking must be flawed).

Could be related to: https://github.com/obsproject/obs-studio/issues/6674#issue-1290654359

stephematician commented 1 year ago

This is an issue with registering a callback (such as load_filter) which tries to enter a graphics context within a (button's) callback. OBS's graphic thread may acquire a lock on the graphic mutex and then attempt to render the lua source - which attempts to acquire a lock on the lua script. Concurrently, the button press will acquire a lock on the lua script, and then attempt to enter a graphic context (acquire the graphic mutex) as in load_filter.

Added this info to https://github.com/obsproject/obs-studio/issues/6674#issuecomment-1411912988