MediaGun / QuickVLC

Vlc and Qt (Qml) integration library
Other
8 stars 2 forks source link

Unexpected application crash #1

Closed aa-bamboo closed 2 years ago

aa-bamboo commented 2 years ago

The library renders video frames using callbacks (libvlc_video_set_output_callbacks from libvlc v4.0.0) into OpenGL framebuffer. Then framebuffer's texture is used for QSGTexture object.

We use the last version of LibVLC v4.0.0 from "master" branch: nightly builds for Windows and MacOS X and daily recompiled from source version for Linux.

The main trouble now is a crash when we try to play one file after the previous one. The crash occurs unexpectedly.

Call-stack doesn't provide enough information (something with threads, all debug symbols are installed):

    1 ??                                0x7fff7d346264
    2 ??                                0x7fff7cdbb37f
    3 ??                                0x7fff7cdd1e96
    4 ??                                0x7fff7cde4fcb
    5 ??                                0x7fff7cde626d
    6 ??                                0x7fff7cdedd68
    7 ??                                0x7fff7d090eab
    8 start_thread pthread_create.c 477 0x7ffff5ea3609
    9 clone        clone.S          95  0x7ffff6a09163 
chubinou commented 2 years ago

Hi,

On which platform do you observe the issue?

Do you only observe it on next? or on sometime while opening the first media

Do you still have the issue if you use EGL backend of Qt (exporting QT_XCB_GL_INTEGRATION=xcb_egl) and remove the "eglQueryString" hack

Do you still have the issue if you disable accelerated hardware with --no-hw-dec

aa-bamboo commented 2 years ago

On which platform do you observe the issue?

The issue exist on all three platforms: Windows, Linux, MacOS X.

Do you only observe it on next? or on sometime while opening the first media

Crash usually occurs on next file. But sometimes on first one.

Do you still have the issue if you use EGL backend of Qt (exporting QT_XCB_GL_INTEGRATION=xcb_egl) and remove the "eglQueryString" hack.

Yes it crashes. I tried this proposal and application crashes. However, according to the forum's response additional env parameter QT_XCB_GL_INTEGRATION is not required. But after changes you proposed applicatoin starts without "eglQueryString" hack.

Do you still have the issue if you disable accelerated hardware with --no-hw-dec

Yes.

chubinou commented 2 years ago
chubinou commented 2 years ago

by the way, if you want to check whether Qt is using EGL or not you can retrieve its context (it should be non-null when using EGL)

///m_context = new QOpenGLContext(this);
QPlatformNativeInterface *nativeInterface = QGuiApplication::platformNativeInterface();
bool isUsingEGL = nativeInterface->nativeResourceForContext("eglContext", m_context) != nullptr;

Note that I'm not really sure how the QOpenGLContext works now with Qt6 on windows and OSX, In Qt5, it use to leverage ANGLE on windows which is an EGL implementation on top of D3D11, I but as far as I know they removed ANGLE, so they probably use the system OpenGL implementation which is WGL, but I don't know how does it interact with their QML backend which should use D3D (unless you force it to use OpenGL)

aa-bamboo commented 2 years ago

what version of VLC did you you test (which hash or the date of the nightly)

The latest. I rebuild master branch from gitlab repo for linux daily. Also I update windows library from nightly builds.

Debug info:

Using libvlc version: "4.0.0-dev Otto Chriek"
Volume set 0.399994
libva info: VA-API version 1.7.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
[00007fff10119450] vaapi generic: Using Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 ()

Stacktrace Strange. Now I have segfault in different place...

1  QOpenGLFramebufferObject::texture                                                                                                                                                             qopenglframebufferobject.cpp 1235 0x7fffe4206390 
2  Vlc::VideoFrame::VideoFrame                                                                                                                                                                   videoframe.cpp               26   0x7fff62beea48 
3  __gnu_cxx::new_allocator<Vlc::VideoFrame>::construct<Vlc::VideoFrame, QOpenGLFramebufferObject *>                                                                                             new_allocator.h              146  0x7fff62bee4e5 
4  std::allocator_traits<std::allocator<Vlc::VideoFrame>>::construct<Vlc::VideoFrame, QOpenGLFramebufferObject *>                                                                                alloc_traits.h               483  0x7fff62bee3c4 
5  std::_Sp_counted_ptr_inplace<Vlc::VideoFrame, std::allocator<Vlc::VideoFrame>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<QOpenGLFramebufferObject *>                               shared_ptr_base.h            548  0x7fff62bee0ce 
6  std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<Vlc::VideoFrame, std::allocator<Vlc::VideoFrame>, QOpenGLFramebufferObject *>                                                 shared_ptr_base.h            679  0x7fff62bedcfe 
7  std::__shared_ptr<Vlc::VideoFrame, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<Vlc::VideoFrame>, QOpenGLFramebufferObject *>                                                     shared_ptr_base.h            1344 0x7fff62bedaca 
8  std::shared_ptr<Vlc::VideoFrame>::shared_ptr<std::allocator<Vlc::VideoFrame>, QOpenGLFramebufferObject *>                                                                                     shared_ptr.h                 359  0x7fff62bed7cf 
9  std::allocate_shared<Vlc::VideoFrame, std::allocator<Vlc::VideoFrame>, QOpenGLFramebufferObject *>                                                                                            shared_ptr.h                 702  0x7fff62bed3cd 
10 std::make_shared<Vlc::VideoFrame, QOpenGLFramebufferObject *>                                                                                                                                 shared_ptr.h                 718  0x7fff62becdfb 
11 Vlc::OpenGLVideoStream::getVideoFrame                                                                                                                                                         openglvideostream.cpp        70   0x7fff62bec128 
12 VideoStream::frameUpdated                                                                                                                                                                     videostream.cpp              65   0x7fff78441df5 
13 QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, void (Vlc::OpenGLVideoStream:: *)()>::call(void (Vlc::OpenGLVideoStream:: *)(), Vlc::OpenGLVideoStream *, void * *) qobjectdefs_impl.h           171  0x7fff62bede63 
14 QtPrivate::FunctionPointer<void (Vlc::OpenGLVideoStream:: *)()>::call<QtPrivate::List<>, void>(void (Vlc::OpenGLVideoStream:: *)(), Vlc::OpenGLVideoStream *, void * *)                       qobjectdefs_impl.h           208  0x7fff62bedbf0 
15 QtPrivate::QSlotObject<void (Vlc::OpenGLVideoStream:: *)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase *, QObject *, void * *, bool *)                                    qobjectdefs_impl.h           419  0x7fff62bed96b 
16 QObject::event                                                                                                                                                                                qobject.cpp                  1356 0x7ffff245ea7c 
17 QApplicationPrivate::notify_helper                                                                                                                                                            qapplication.cpp             3337 0x7ffff341e432 
18 QCoreApplication::notifyInternal2                                                                                                                                                             qthread_p.h                  316  0x7ffff240dfba 
19 QCoreApplicationPrivate::sendPostedEvents                                                                                                                                                     qcoreapplication.cpp         1848 0x7ffff2410edc 
20 postEventSourceDispatch                                                                                                                                                                       qeventdispatcher_glib.cpp    279  0x7ffff26b14c3 
21 g_main_context_dispatch                                                                                                                                                                                                         0x7fffef91317d 
22 ??                                                                                                                                                                                                                              0x7fffef913400 
23 g_main_context_iteration                                                                                                                                                                                                        0x7fffef9134a3 
24 QEventDispatcherGlib::processEvents                                                                                                                                                           qeventdispatcher_glib.cpp    429  0x7ffff26b0e4a 
25 QEventLoop::exec                                                                                                                                                                              qflags.h                     144  0x7ffff241a1eb 
26 QCoreApplication::exec                                                                                                                                                                        qflags.h                     110  0x7ffff2415dae 
27 main                                                                                                                                                                                          main.cpp                     117  0x5555555e1737 
chubinou commented 2 years ago

Strange. Now I have segfault in different place...

you should probably take the m_text_lock during resize and cleanup, this looks like a use after free.

The latest. I rebuild master branch from gitlab repo for linux daily

hum, Ok, but that's a bit harder to know whether I'm investigating the same bug or not with a moving target, code has changed in VLC since your report post on the forum, notably in the VA-API part.

To be clear: I do reproduce a crash with your code (and our sample QtOpenGL app), but it's working working correctly when forcing egl and when disabling hw-acceleration, not saying that you should not use glx, but I wonder if we're looking at the same issue.

debug info [...]

you need to add "-vvvv" in libvlc arguments.

by the way: using in instance.cpp the line argv[i] = static_cast<char *>(args.at(i).toUtf8().data()); is unsafe as toUtf8 will create a temporary QString that will be destroyed when leaving the assignation, so your pointer points to an unassigned memory space. You should probably do the qstrdup like you do on windows

aa-bamboo commented 2 years ago

Thank you. Let me apply suggestions. Also I'll share verbose log and synchronize library version across every platform, then I'll share that version number with you to be using the same build.

chubinou commented 2 years ago

Hi, can you try to build VLC with this patch ?

https://code.videolan.org/videolan/vlc/-/merge_requests/2017

aa-bamboo commented 2 years ago

Hi, can you try to build VLC with this patch ?

https://code.videolan.org/videolan/vlc/-/merge_requests/2017

I'll try

aa-bamboo commented 2 years ago

you should probably take the m_text_lock during resize and cleanup, this looks like a use after free.

Adding lock m_text_lock into both resize and cleanup causes deadlock. Adding it separately into resize or cleanup doesn't take any effect.

To be clear: I do reproduce a crash with your code (and our sample QtOpenGL app), but it's working working correctly when forcing egl and when disabling hw-acceleration, not saying that you should not use glx, but I wonder if we're looking at the same issue.

It seems that it is the same issue.

you need to add "-vvvv" in libvlc arguments.

I've added it into arguments. But there is full log:

VLC ARGS: QList("-vvvv")
Using libvlc version: "4.0.0-dev Otto Chriek"
qml: playpreview
Volume set 0.399994
libva info: VA-API version 1.7.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
[00007fec54119790] vaapi generic: Using Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 ()
[00007fec541609d0] gl gl error: Could not create interop
[00007fec5406b330] main video output error: video output display creation failed
QSGTexture anisotropy levels are not currently supported
Texture 0x7fecd0a27c10 () used with different accesses within the same pass, this is not allowed.
Texture 0x7fecd08cd1a0 () used with different accesses within the same pass, this is not allowed.
qml: playpreview
[h264 @ 0x7fec701488c0] Discarding mismatching reference
[h264 @ 0x7fec701488c0] Discarding mismatching reference
[h264 @ 0x7fec701488c0] Missing reference picture, default is 0
[h264 @ 0x7fec701488c0] decode_slice_header error
[00007fec4001f950] vaapi generic: Using Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 ()
[00007fec40005ce0] gl gl error: Could not create interop
[00007fec5406b330] main video output error: video output display creation failed

by the way: using in instance.cpp the line argv[i] = static_cast<char *>(args.at(i).toUtf8().data()); is unsafe as toUtf8 will create a temporary QString that will be destroyed when leaving the assignation, so your pointer points to an unassigned memory space. You should probably do the qstrdup like you do on windows

Ok. Let me make changes.

chubinou commented 2 years ago

Adding lock m_text_lock into both resize and cleanup causes deadlock

yes obviously, as resize calls cleanup. In your case there isn't much value in calling cleanup as the texture will be released when reassigning the unique_ptr

for what it's worth I put my local changes on this branch https://github.com/chubinou/QuickVLC/commits/hotfixes

It seems that it is the same issue.

So ... does the patch I sent fixes your issue?

I've added it into arguments. But there is full log:

hum, that log is still incomplete, I suspect that this is linked to the issue with argv[i] = static_cast<char *>(args.at(i).toUtf8().data());

A complete log should look like this https://pastebin.com/G1a6rxXF

Now the upcomming issue it that you don't have hw accceleration with GLX (so on linux when Qt run with GLX backend), as we don't have an "interop" between VAAPI (the hw decoder) and GLX, only the EGL one is available at the moment.

aa-bamboo commented 2 years ago

Hi, can you try to build VLC with this patch ?

https://code.videolan.org/videolan/vlc/-/merge_requests/2017

The patch hasn't fixed the crash.

I started to play 4 videos one by one. And on the 4th file application has crashed.

The full log is here (I added args fix): LibVLC log

Next I'm going to apply and check changes from hotfixes branch.

aa-bamboo commented 2 years ago

for what it's worth I put my local changes on this branch https://github.com/chubinou/QuickVLC/commits/hotfixes

I checked and tested changes multiple times with different files. And there were no more crashes. Seems that issue has fixed. Let me clean up the code, and then I'll share the final results.

aa-bamboo commented 2 years ago

Update.

The patch https://code.videolan.org/videolan/vlc/-/merge_requests/2017 fixes crashes on some specific videos with specific codecs. Also it fixes "black screen" on some dvd's videos.

On other side changes from hotfix branch fix the crash while starting different files one by one.

So we need to apply both changes: from patch and from hotfixes branch.

chubinou commented 2 years ago

Seems that issue has fixed.

OK good news

So we need to apply both changes

for your information, the MR 2017 has been merged upstream

aa-bamboo commented 2 years ago

Thank you. Yes, I pulled an upstream