kkartaltepe
commented
2 months ago Please include the stack trace
Closed alefminus closed 1 month ago
kkartaltepe
commented
2 months ago Please include the stack trace
alefminus
commented
2 months ago Right, obvious oversight, my bad. When trying to reproduce of course suddenly I could not do it with gdb (difference is this time I opted out of the symbol download), but lldb complied, here it is with lldb, same steps:
Unsupported feature/library load error. /builddir/build/BUILD/obs-studio-30.2.2/plugins/obs-qsv11/QSV_Encoder_Internal.cpp 160
warning: [qsv encoder: 'msdk_impl'] Unsupported configurations, parameters, or features (MFX_ERR_UNSUPPORTED)
Process 178152 stopped
* thread #1, name = 'obs', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x4f178)
frame #0: 0x00007fffd201b856 obs-qsv11.so`simple_free(void*, mfxFrameAllocResponse*) + 150
obs-qsv11.so`simple_free:
-> 0x7fffd201b856 <+150>: movq (%rax,%rdx,8), %r15
0x7fffd201b85a <+154>: movq %rbx, %rax
0x7fffd201b85d <+157>: rep stosq %rax, %es:(%rdi)
0x7fffd201b860 <+160>: callq 0x7fffd20192c0 ; ___lldb_unnamed_symbol448 + 2288
(lldb) bt
* thread #1, name = 'obs', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x4f178)
* frame #0: 0x00007fffd201b856 obs-qsv11.so`simple_free(void*, mfxFrameAllocResponse*) + 150
frame #1: 0x00007fffd201efc1 obs-qsv11.so`QSV_Encoder_Internal::ClearData() + 177
frame #2: 0x00007fffd201f1af obs-qsv11.so`QSV_Encoder_Internal::~QSV_Encoder_Internal() + 31
frame #3: 0x00007fffd201f29b obs-qsv11.so`qsv_encoder_open + 155
frame #4: 0x00007fffd20201f7 obs-qsv11.so`obs_qsv_create + 1463
frame #5: 0x00007fffd2020d5a obs-qsv11.so`obs_qsv_create_tex + 202
frame #6: 0x00007ffff6b3efbb libobs.so.0`obs_encoder_initialize + 747
frame #7: 0x00007ffff6b58ab1 libobs.so.0`obs_output_initialize_encoders + 177
frame #8: 0x00007fffbb4b7dfa obs-ffmpeg.so`ffmpeg_mux_start.lto_priv.0 + 250
frame #9: 0x00007ffff6b5531b libobs.so.0`obs_output_actual_start + 91
frame #10: 0x00007ffff6b5584c libobs.so.0`obs_output_start + 76
frame #11: 0x000055555570de2b obs`SimpleOutput::StartRecording() + 59
frame #12: 0x00007ffff35fc872 libQt6Core.so.6`void doActivate<false>(QObject*, int, void**) + 1394
frame #13: 0x00007ffff35fc872 libQt6Core.so.6`void doActivate<false>(QObject*, int, void**) + 1394
frame #14: 0x00007ffff4ac3114 libQt6Widgets.so.6`QAbstractButton::clicked(bool) + 68
frame #15: 0x00007ffff4ac339f libQt6Widgets.so.6`QAbstractButtonPrivate::emitClicked() + 63
frame #16: 0x00007ffff4ac4a7f libQt6Widgets.so.6`QAbstractButtonPrivate::click() + 207
frame #17: 0x00007ffff4ac4c95 libQt6Widgets.so.6`QAbstractButton::mouseReleaseEvent(QMouseEvent*) + 277
frame #18: 0x00007ffff49ed158 libQt6Widgets.so.6`QWidget::event(QEvent*) + 696
frame #19: 0x00007ffff498b218 libQt6Widgets.so.6`QApplicationPrivate::notify_helper(QObject*, QEvent*) + 136
frame #20: 0x00007ffff4996142 libQt6Widgets.so.6`QApplication::notify(QObject*, QEvent*) + 3698
frame #21: 0x00007ffff3596dc8 libQt6Core.so.6`QCoreApplication::notifyInternal2(QObject*, QEvent*) + 376
frame #22: 0x00007ffff49946cc libQt6Widgets.so.6`QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool) + 492
frame #23: 0x00007ffff4a011f6 libQt6Widgets.so.6`QWidgetWindow::handleMouseEvent(QMouseEvent*) + 902
frame #24: 0x00007ffff4a04130 libQt6Widgets.so.6`QWidgetWindow::event(QEvent*) + 512
frame #25: 0x00007ffff498b218 libQt6Widgets.so.6`QApplicationPrivate::notify_helper(QObject*, QEvent*) + 136
frame #26: 0x00007ffff3596dc8 libQt6Core.so.6`QCoreApplication::notifyInternal2(QObject*, QEvent*) + 376
frame #27: 0x00007ffff3de6eab libQt6Gui.so.6`QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) + 1995
frame #28: 0x00007ffff3e4b4bc libQt6Gui.so.6`QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) + 172
frame #29: 0x00007ffff43633f4 libQt6Gui.so.6`userEventSourceDispatch(_GSource*, int (*)(void*), void*) + 20
frame #30: 0x00007ffff2d1de8c libglib-2.0.so.0`g_main_context_dispatch_unlocked.lto_priv.0 + 316
frame #31: 0x00007ffff2d7fc98 libglib-2.0.so.0`g_main_context_iterate_unlocked.isra.0 + 728
frame #32: 0x00007ffff2d1f383 libglib-2.0.so.0`g_main_context_iteration + 51
frame #33: 0x00007ffff38850a3 libQt6Core.so.6`QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) + 115
frame #34: 0x00007ffff35a3b03 libQt6Core.so.6`QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) + 435
frame #35: 0x00007ffff359f9bc libQt6Core.so.6`QCoreApplication::exec() + 156
frame #36: 0x0000555555604c76 obs`main + 6198
frame #37: 0x00007ffff2e39088 libc.so.6`__libc_start_call_main + 120
frame #38: 0x00007ffff2e3914b libc.so.6`__libc_start_main@@GLIBC_2.34 + 139
frame #39: 0x0000555555607f35 obs`_start + 37
Operating System Info
Other
Other OS
fedora 40
OBS Studio Version
30.2.2
OBS Studio Version (Other)
No response
OBS Studio Log URL
https://obsproject.com/logs/dINNx5qpKFWUnkSi
OBS Studio Crash Log URL
No response
Expected Behavior
Records stream.
Current Behavior
Crashes with a segmentation violation.
Steps to Reproduce
Anything else we should know?
The log fails to show the actual segmentation violation, here it is when running under gdb, including a stack trace and faulted relevant variables:
So it seems as I wrote in the title that response is valid but response->mids is getting an invalid address, perhaps some offset from a NULL that was unchecked.