Black screen with gstreamer as Qt Media backend (Recent KDE Neon update)

linlinxza commented 7 months ago

This is what I get according to journalctl -f:

Failed to initialize QMediaPlayer "Could not find the videoconvertscale GStreamer element"

Yes, I have gstreamer1.0-libav installed.

linlinxza commented 7 months ago

I didn't install anything as far as I recall. And how do I get to this plugins/multimedia/ directory to run ldd libffmpegmediaplugin.so | grep libavformat? By going to /usr/share/plugins/multimedia/? How do I find the complete path?

dpkg -L libva2 libva-wayland2 libva-x11-2 libva-dev output:

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libva.so.2.1400.0
/usr/share
/usr/share/doc
/usr/share/doc/libva2
/usr/share/doc/libva2/changelog.Debian.gz
/usr/share/doc/libva2/copyright
/usr/lib/x86_64-linux-gnu/libva.so.2

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libva-wayland.so.2.1400.0
/usr/share
/usr/share/doc
/usr/share/doc/libva-wayland2
/usr/share/doc/libva-wayland2/copyright
/usr/lib/x86_64-linux-gnu/libva-wayland.so.2
/usr/share/doc/libva-wayland2/changelog.Debian.gz

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libva-x11.so.2.1400.0
/usr/share
/usr/share/doc
/usr/share/doc/libva-x11-2
/usr/share/doc/libva-x11-2/copyright
/usr/lib/x86_64-linux-gnu/libva-x11.so.2
/usr/share/doc/libva-x11-2/changelog.Debian.gz

dpkg-query: package 'libva-dev' is not installed
Use dpkg --contents (= dpkg-deb --contents) to list archive files contents.
lindsey@Lindsey-PC:~$ dpkg -L libva2 libva-wayland2 libva-x11-2 libva-dev
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libva.so.2.1400.0
/usr/share
/usr/share/doc
/usr/share/doc/libva2
/usr/share/doc/libva2/changelog.Debian.gz
/usr/share/doc/libva2/copyright
/usr/lib/x86_64-linux-gnu/libva.so.2

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libva-wayland.so.2.1400.0
/usr/share
/usr/share/doc
/usr/share/doc/libva-wayland2
/usr/share/doc/libva-wayland2/copyright
/usr/lib/x86_64-linux-gnu/libva-wayland.so.2
/usr/share/doc/libva-wayland2/changelog.Debian.gz

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libva-x11.so.2.1400.0
/usr/share
/usr/share/doc
/usr/share/doc/libva-x11-2
/usr/share/doc/libva-x11-2/copyright
/usr/lib/x86_64-linux-gnu/libva-x11.so.2
/usr/share/doc/libva-x11-2/changelog.Debian.gz

dpkg-query: package 'libva-dev' is not installed
Use dpkg --contents (= dpkg-deb --contents) to list archive files contents.

vainfo output:

libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.14 (libva 2.12.0)
vainfo: Driver version: Mesa Gallium driver 23.2.1-1ubuntu3.1~22.04.2 for AMD Radeon RX 580 Series (polaris10, LLVM 15.0.7, DRM 3.54, 6.5.0-28-generic)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

QT_FFMPEG_DEBUG=1 QSG_INFO=1 QT_LOGGING_RULES="*.debug=true" /usr/lib/qt6/bin/qml test.qml test.qml 2> output.txt output: output.txt

luisbocanegra commented 7 months ago

I didn't install anything as far as I recall. And how do I get to this plugins/multimedia/ directory to run ldd libffmpegmediaplugin.so | grep libavformat?

According to ubuntu package database it should be in /usr/lib/x86_64-linux-gnu/qt6/plugins/multimedia/libffmpegmediaplugin.so

linlinxza commented 7 months ago

Output of ldd libffmpegmediaplugin.so | grep libavformat:

libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007ba2a2800000)

larsut commented 6 months ago

Thanks a lot, @linlinxza!

I’m still unable to reproduce this crash, even when using FFmpeg 4.4.2 installed with “sudo apt install ffmpeg” and preventing Qt from dynamically loading VAAPI, which is happening in your logs. However I’m not using KDE Plasma.

Two things that could be checked:

Run the qml app with LIBVA_DRIVER_NAME=nothing , which prevents both Qt and FFmpeg from using VAAPI. I’m expecting that to work fine without crashing.
Check FFmpeg version with the command “FFmpeg -version”

@luisbocanegra
I’ll close the Qt bug report, as this now seems to be an issue with FFmpeg / VA-API. We can re-open it if new info points in the direction of Qt. I submitted a patch which helps Qt load VAAPI if FFmpeg can find it, but I’m not sure if it will affect this bug. If you wan’t to investigate this further, here's a version of the latest debug output, starting when QML is switching to the last video playing before it crashes, and with non-relevant stuff filtered out. QTBUG-124586-crashlog-filtered_last-part.txt

Here are some clues:

Line 550 in the user log states "FFmpeg log: Direct mapping disabled: deriving image does not work: 1 (operation failed).”, meaning FFmpeg gets an error when calling vaDeriveImage() from hwcontext_vaapi.c. My log says “"FFmpeg log: Direct mapping possible”.

Just before crashing with “corrupted size vs. prev_size”, FFmpeg is decoding to surface 0x15 twice in a row, on line 673 and 682. I think this can happen in a normal situation, but if you inspect the “Decode to surface” lines, it seems that there are two different threads performing decode operations on the same surfaces, which would be a plausible explanation for the “corrupted size vs. prev_size” crash.

luisbocanegra commented 6 months ago

@larsut Thanks your all your efforts, unfortunately I am not familiar with the languages and video/graphics stuff.

For now I will wait for @linlinxza to get the fix from an update and hopefully that fixes the problem. If it does not, I would like them to first try with another rolling distribution to discard this being a packaging bug of some sort or just the consequence of not very up-to-date packages from Ubuntu.

linlinxza commented 6 months ago

Still not working.... And I'm completely up to date.

And I really don't want to switch distros just because I have everything the way I want it. I've been literally using the same distro for over 5 years now.

larsut commented 6 months ago

@linlinxza I'm sorry to hear. If you have time, you could try these steps that I mentioned:

Two things that could be checked:

Run the qml app with LIBVA_DRIVER_NAME=nothing , which prevents both Qt and FFmpeg from using VAAPI. I’m expecting that to work fine without crashing.

Check FFmpeg version with the command “FFmpeg -version”

linlinxza commented 6 months ago

My FFmpeg version is: 4.4.2-0ubuntu0.22.04.1

Anyway, I'll try the other stuff tomorrow. It's like 5 AM where I am. lol

I just came on here to comment because the rest of the Internet is largely useless (namely with social media); much of which I blocked via my PiHole and used filters on Ublock (that I personally devised) to remove any and all references to social media domains in the search engine and also share buttons. Feels like early 2000's Internet before social media took off with a lot less crap. :)

linlinxza commented 5 months ago

@linlinxza I'm sorry to hear. If you have time, you could try these steps that I mentioned:

Two things that could be checked:

Run the qml app with LIBVA_DRIVER_NAME=nothing , which prevents both Qt and FFmpeg from using VAAPI. I’m expecting that to work fine without crashing.

Check FFmpeg version with the command “FFmpeg -version”

UPDATE:

You are my hero! I think this is working. I have been able to have the video background that is a minute long loop for about 10 minutes now without a single crash with plasmashell!

I created the file ~/.config/plasma-workspace/env/qt-media-backend.sh as per Luisbocanegra's instructions. But I altered it to be like this:

#!/bin/bash
export LIBVA_DRIVER_NAME=nothing

And lo and behold, it frigging works! I think this will have to be the only solution for people with AMD cards who are using distros with older packaging!

EDIT:

But this does pose a performance issue now. Because it's relying on the CPU more and not the GPU. So, AMD users who need to use this method are leaving performance on the table. I have to re-encode the video clips from 1080p to 720p so that it isn't so taxing on the CPU.

Is there another driver name I could specify?

larsut commented 5 months ago

I appreciate your appreciation! Yea this workaround just prevents FFmpeg from using hardware acceleration through VA-API (libva), so the CPU has to do more work. Based on your logs FFmpeg seemed to be using the correct driver before, so (installing) and specifying another driver shouldn't help.

However, I got a new idea. Can you test if these three slightly edited qml test apps also crash? Don't use LIBVA_DRIVER_NAME=nothing for these tests.

Video_test-1_edited_versions.zip

larsut commented 5 months ago

@luisbocanegra I wanted to include a test version without "loops: MediaPlayer.Infinite" as well, but that unfortunately leads to a crash because of another bug. Will fix that one asap. However the call to stop() before playNext() should have the same effect. This could prevent the sketchy behaviour I described earlier, where FFmpeg seems to start playing the next video on a new thread using VAAPI hardware texture surfaces that are still in use by the old video thread.

larsut commented 5 months ago

@linlinxza Test this file first of all. @luisbocanegra it uses the QML Types MediaPlayer and VideoOutput instead of just Video. This makes it possible to listen for onMediaStatusChanged instead of onPositionChanged. EndOfMedia status is emitted later than positionChanged, and that could prevent this whole VAAPI issue if we are lucky. video_test-1_without_looping_and_with_mediaplayer.qml.zip

luisbocanegra commented 5 months ago

@linlinxza Thank you for your patience helping us troubleshoot this issue, it is really appreciated.

it uses the QML Types MediaPlayer and VideoOutput instead of just Video. This makes it possible to listen for onMediaStatusChanged instead of onPositionChanged. EndOfMedia status is emitted later than positionChanged, and that could prevent this whole VAAPI issue if we are lucky. video_test-1_without_looping_and_with_mediaplayer.qml.zip

@larsut Thank you for continuing investigating this issue, hopefully that prevents the problem.

I wanted to include a test version without "loops: MediaPlayer.Infinite" as well, but that unfortunately leads to a crash because of another bug.

Is this bug something that could be platform specific or is also related to AMD? I have tried removing it from the Video type in video_test-1_with_stop and it still works, removing it from video_test-1_without_looping_and_with_mediaplayer.qml also works and doing so allows looping through all the videos.

linlinxza commented 5 months ago

You are welcome, luisboanegra.

Here's what I have for logs (did these on mom's AMD laptop):

with-delay-log (qml debug enabled).txt with-stop-and-delay-log (qml debug enabled).txt with-stop-log (qml debug enabled).txt

Interestingly enough, the test files would go through all the videos if I don't have qml debugging enabled if I run it like this:

/usr/lib/qt6/bin/qml video_test-1_with_delay.qml

Instead of like this this:

QSG_INFO=1 QT_LOGGING_RULES="qml.debug=true" /usr/lib/qt6/bin/qml video_test-1_with_delay.qml

Despite getting this issue after each clip:

[AVHWFramesContext @ 0x7f5c180051c0] Failed to sync surface 0x15: 13 (the requested VAEntryPoint is not supported).
Error transferring the data to system memory: -5
Cannot map a video frame in ReadOnly mode!
Cannot represent the plane 0 as an image
Cannot represent the plane 1 as an image
vaExportSurfaceHandle failed

The next clip will play. It won't crash.

luisbocanegra commented 5 months ago

Please try the mediaplayer one https://github.com/luisbocanegra/plasma-smart-video-wallpaper-reborn/issues/8#issuecomment-2135328373

luisbocanegra commented 5 months ago

Switched the wallpaper from Video to Mediaplayer https://github.com/luisbocanegra/plasma-smart-video-wallpaper-reborn/commit/793ecab26d078373297ce028ef0187c0fc4bf74f in https://github.com/luisbocanegra/plasma-smart-video-wallpaper-reborn/tree/switch-to-mediaplayer branch.

Things to note:

Checking for status == MediaPlayer.EndOfMedia is always false if loops: MediaPlayer.Infinite is set
Removing loops property (defaults to 1) allows looping through all the videos but when there is only one video the loop is not smooth

So I changed loops to be infinite when there's only one video and 1 otherwise

@linlinxza please download the file below and run kpackagetool6 -t Plasma/Wallpaper -u plasma-smart-video-wallpaper-reborn-v0.2.1.zip, reboot/re-login and try enabling it with a single video first then multiple.

plasma-smart-video-wallpaper-reborn-v0.2.1.zip

linlinxza commented 5 months ago

I think you are getting somewhere. If I add just one video, plasmashell will crash. But if I add two video clips, they will keep rotating without crashing plasmashell. I tried this on my mom's laptop which doesn't have the config file ~/.config/plasma-workspace/env/qt-media-backend.sh workaround. So far, if I put two copies of the same video (naming them 1 and 2 respectively to have the same recurring clip) in and have them playing, it works perfectly.

PS; I didn't have to reboot, or log off and log back in again. I just changed the wallpaper plugin to "Image" and then changed it back to your plugin after upgrading it.

I then tried this on my PC and I removed that qt-media-backend.sh file (and of course logged off for that). It's been working so far without taxing my CPU with 1080p video now.

luisbocanegra commented 5 months ago

Interesting...

Try the following version

plasma-smart-video-wallpaper-reborn-v0.2.1-loops1.zip

luisbocanegra commented 5 months ago

Interesting...

Try the following version

plasma-smart-video-wallpaper-reborn-v0.2.1-loops1.zip

Note This version will show the background color (black by default) for a split of a second when there is only one video because it isn't looping MediaPlayer.Infinite but instead repeating (or playing the next) when the current one ends.

linlinxza commented 5 months ago

Note This version will show the background color (black by default) for a split of a second when there is only one video because it isn't looping MediaPlayer.Infinite but instead repeating (or playing the next) when the current one ends.

It does the same thing already with the previous revision; as in a black background color with what I have now (by using two of the same clips).

EDIT:

Anyway, I tried the 0.2.1-loops1 version. This works without having to add the copy of the same clip. Now, if it could be smoothed out so it doesn't show any black background, then this would be great.

luisbocanegra commented 5 months ago

Anyway, I tried the 0.2.1-loops1 version. This works without having to add the copy of the same clip. Now, if it could be smoothed out so it doesn't show any black background, then this would be great.

Nice, it means @larsut is right and the problem is on the ffmpeg loop implementation.

Out of curiosity can you try this one. It's basically what we started with but with MediaPlayer instead of Video type. I expect it to also crash but doesn't hurt trying

plasma-smart-video-wallpaper-reborn-v0.2.1-loopsInfiniteOnPosition.zip

linlinxza commented 5 months ago

Nice, it means @larsut is right and the problem is on the ffmpeg loop implementation.

Out of curiosity can you try this one. It's basically what we started with but with MediaPlayer instead of Video type. I expect it to also crash but doesn't hurt trying

plasma-smart-video-wallpaper-reborn-v0.2.1-loopsInfiniteOnPosition.zip

Well, just tried it. So far, it hasn't crashed plasmashell yet. It just does the same thing as the last one; which is a black background for split second before playing the clip again.

luisbocanegra commented 5 months ago

For a single video? For me the black background only shows for multiple videos with that one

linlinxza commented 5 months ago

For a single video? For me the black background only shows for multiple videos with that one

Well, I could try it on another machine with AMD graphics. But, I am sure the result will be the same. Speaking of, got a machine with AMD graphics on it yet?

luisbocanegra commented 5 months ago

For a single video? For me the black background only shows for multiple videos with that one

Well, I could try it on another machine with AMD graphics. But, I am sure the result will be the same. Speaking of, got a machine with AMD graphics on it yet?

Nope

linlinxza commented 5 months ago

Well, I could try it on another machine with AMD graphics. But, I am sure the result will be the same. Speaking of, got a machine with AMD graphics on it yet?

Nope

Even a cheap old machine with AMD graphics would do it. You need to have access to the that kind of hardware so that you could better troubleshoot. Because I can only do so much (given my lack of comprehending how to read and write code; and not knowing how it works). Best I can do is simple 'if' and 'while' statements.

linlinxza commented 5 months ago

Well, it was great while it lasted. But after rebooting and then updating, plasmashell crashes again with it. And now the background goes green instead of black again...

And if I try putting ~/.config/plasma-workspace/env/qt-media-backend.sh back to gstreamer, the video stays frozen. Like still picture.

EDIT:

This is so damn weird... it works again. I tried setting ~/.config/plasma-workspace/env/qt-media-backend.sh to ffmpeg as a test. And then, after logging off and logging in again, it works again with no crashing yet.

I am totally lost on why this thing works and then doesn't. Well, guess I'll enjoy it while I have it.

linlinxza commented 2 months ago

I don't know what happened, but the plugin seems to be working now. I tried it on both computers that use AMD graphics. It works. The video has looped every time so far without crashing plasmashell from what I noticed.

The screen goes black for a split second, but still keeps on successfully keeps looping the same video file.

luisbocanegra commented 2 months ago

Nice! You can check the packages that have been updated recently either qt (specially qt-multimedia), ffmpeg, plasma, gpu drivers...

The screen goes black for a split second, but still keeps on successfully keeps looping the same video file.

Does the crossfade option work?

linlinxza commented 2 months ago

Fade doesn't work. Also, I don't know how to check anything. All I do is just keep my system updated regularly.

luisbocanegra / plasma-smart-video-wallpaper-reborn

Black screen with gstreamer as Qt Media backend (Recent KDE Neon update) #8