Raspberry Pi OS 64 Bit

[sagiadinos]

Hello,

Does anyone have experiences with Raspberry Pi?

Compiling qtavplayer on Raspberry Pi OS 64 Bit fails: compile.log is empty Summary is The following modules are not compiled in this configuration QtAVPlayer

Assuming that there goes something wrong with acceleration as pi uses V4L2 codec APIs for hw decoding.

Any plans to support FFmpeg v4l2 API?

It is possible to compile qtavplayer with FFmpeg software decoding on pi?

Thank you in advance

[valbok]

Hello,

Summary is The following modules are not compiled in this configuration QtAVPlayer

can you check that you have QtMultimedia installed?

[sagiadinos]

Hello Val,

Not it wasn't. Thank you. Now it starts to compile. But there are other issues.

First: you need to include /usr/include/c++/10 in $CPLUS_INCLUDE_PATH to prevent this error:

/usr/include/c++/10/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
   75 | #include_next <stdlib.h>

So, I use this bash script. Qt is the repository Qt5.15.2 from the OS

#!/bin/bash
set -e
export LIBRARY_PATH=/usr/lib:/usr/lib/aarch64-linux-gnu:$LIBRARY_PATH
export CPLUS_INCLUDE_PATH=/usr/include/c++/10:/usr/include/aarch64-linux-gnu:$CPLUS_INCLUDE_PATH
mkdir -p build-QtAVPlayer
cd build-QtAVPlayer
qmake /home/pi/QtAVPlayer/QtAVPlayer.pro
make -j4

Now this error came after building the libQt5AVPlayer.so correctly:

Project ERROR: Unknown module(s) in QT: qtmultimediaquicktools-private
make[1]: *** [Makefile:49: sub-qml_video-make_first] Error 3
make[1]: *** Waiting for unfinished jobs....
make[2]: Entering directory '/home/pi/build-QtAVPlayer/examples/extract_frames'

Would it be possible to exclude the build of the example's directory via command?

I installed qml-module-qtquick-privatewidgets qml-module-qtmultimedia and some others, but nothing changed. The alternative was for me to rename the example directory.

[valbok]

Project ERROR: Unknown module(s) in QT: qtmultimediaquicktools-private

Did you workaround this? Now works? BTW how are you going to render frames?

did you try qtdeclarative5-private-dev package?

[sagiadinos]

Hello Val,

Did you workaround this? Now works?

Yes, renaming example directory worked.

did you try qtdeclarative5-private-dev package

Yes, this was already installed. Examples compilation failed.

BTW how are you going to render frames?

1080p HD videos run more or less smoothly, but without hardware acceleration.

top shows something about 170% CPU usage with -platform xcb and surprisingly about 200% with -platform eglfs but stable and without leaks for two days now.

When select libVLC as backend for my player, there is something about 30% CPU usage, but it crashes after an hour.

Same videos with ffplay: ~95%

FFmpeg's acceleration is, in general, suboptimal comparing with libVLC, especially for 4K HEVC, on Raspberry Pi, but there is room for improvements. Unfortunately, I am not familiar enough with FFmpeg decoding that I am able to add a new qavhwdevice_v4l2.cpp class.

[valbok]

top shows something about 170% CPU usage with -platform xcb and surprisingly about 200% with -platform eglfs but stable and without leaks for two days now.

Did you confirm that hardware acceleration was used? What output shows? It would show is it software or hardware decoding. If nothing shows, might be possible that always software decoding is used and it is up to ffmpeg.

1080p HD videos run more or less smoothly, but without hardware acceleration.

Does it mean that you used widgets to render? Because QVideoWidget does not fully support hardware rendering. And all CPU will be busy converting frames to RGB format and copying between renders.

I am not familiar enough with FFmpeg decoding that I am able to add a new qavhwdevice_v4l2.cpp class.

Is it output from v4l2 device? It might look that there is no acceleration, but if there is one, QAVVideoBuffer_GPU should handle it, like decoding in GPU. Accelerated rendering it is another issue and fully related to Qt and how you handle it.

[sagiadinos]

Did you confirm that hardware acceleration was used? What output shows? If nothing shows, might be possible that always software decoding is used and it is up to ffmpeg.

170% – 200% CPU usage means sure no hardware decoding.

ffplay -codec:v h264_v4l2m2m .cache/garlic-player-ffmpeg/048dcda286d6bc0970b7f72d3c72f9f0.mp4

has 75% CPU Usage

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.cache/garlic-player-ffmpeg/048dcda286d6bc0970b7f72d3c72f9f0.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41isomavc1
    creation_time   : 2019-09-01T15:04:26.000000Z
  Duration: 00:00:30.16, start: 0.000000, bitrate: 5574 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 5571 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2019-09-01T15:04:26.000000Z
      handler_name    : L-SMASH Video Handler
      encoder         : AVC Coding
[h264_v4l2m2m @ 0x7f640048f0] level=40
[h264_v4l2m2m @ 0x7f640048f0] Using device /dev/video10
[h264_v4l2m2m @ 0x7f640048f0] driver 'bcm2835-codec' on card 'bcm2835-codec-decode' in mplane mode
[h264_v4l2m2m @ 0x7f640048f0] requesting formats: output=H264 capture=YU12

Which means hardware decoding via /dev/video10

but entering

ffplay .cache/garlic-player-ffmpeg/048dcda286d6bc0970b7f72d3c72f9f0.mp4 has 110% CPU usage and returns

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.cache/garlic-player-ffmpeg/048dcda286d6bc0970b7f72d3c72f9f0.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41isomavc1
    creation_time   : 2019-09-01T15:04:26.000000Z
  Duration: 00:00:30.16, start: 0.000000, bitrate: 5574 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 5571 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2019-09-01T15:04:26.000000Z
      handler_name    : L-SMASH Video Handler
      encoder         : AVC Coding

Assuming this means no hw decode and a difference of 45%.

Does it mean that you used widgets to render?

Yes, and I am aware of the suboptimal render acceleration, but that is only over the top.

FFmpeg 4.3.5 uses h264_v4l2m2m for hardware decode and encode. This is suboptimal compared with libVLC, but possible. My guess it that it won't be used automatically like ffplay needs -codec:v h264_v4l2m2m to activate hw decoding.

Is it output from v4l2 device? It might look that there is no acceleration, but if there is one,

Assuming that there is a hw decode in my player, even if it is not recognizable. Can the missing render acceleration of Qt cause +130% CPU in eglfs?

[valbok]

Assuming that there is a hw decode in my player, even if it is not recognizable. Can the missing render acceleration of Qt cause +130% CPU in eglfs?

Easy to test, just avoid any frame rendering and see CPU

[valbok]

ffplay -codec:v h264_v4l2m2m .cache/garlic-player-ffmpeg/048dcda286d6bc0970b7f72d3c72f9f0.mp4

Added https://github.com/valbok/QtAVPlayer/pull/287 and could you please try setInputVideoCodec("h264_v4l2m2m") ?

[sagiadinos]

Hello Val

I have got some weird results.

setInputVideoCodec("h264_v4l2m2m") seems to activate the hardware decoding of the Raspberry Pi. It goes down from 170% to 102% for xcb and eglfs (formerly 200%)

But: the videos are not played smooth anymore. It is like a slide show with dropping frames.

Deactivating setInputVideoCodec("h264_v4l2m2m") raises the CPU load, but videos played smoothly.

Easy to test, just avoid any frame rendering and see CPU

Doing it.

With setInputVideoCodec("h264_v4l2m2m") and deactivated frame rendering, about 5% CPU Without and deactivated frame rendering about 80% CPU

This should be a sign for a working hw acceleration with setInputVideoCodec("h264_v4l2m2m").

Is it possible, that there is a need to adjust some buffers for QVideoOut?

[valbok]

But: the videos are not played smooth anymore. It is like a slide show with dropping frames.

any errors on stdout? possible bug in pts()? when syncing is buggy and not able to sync with pts of audio, could you try to dump qDebug() << frame.pts(); ? Also could you find video without audio?

[awesomeerictech]

This works, worked for me on bullseye version , the best thing is building qt from source for raspberry Pi

[valbok]

This works, worked for me on bullseye version , the best thing is building qt from source for raspberry Pi

works but without hardware acceleration?

[sagiadinos]

possible bug in pts()? when syncing is buggy and not able to sync with pts of audio,

Please excuse my delay. I am on travel and have no Raspberry to test. In about 10 days I will be back in my home lab.

[valbok]

Interesting @geminixdev found a bug https://github.com/valbok/QtAVPlayer/issues/303 which can be relevant also for another platform, when QAVPlayer skips some frames

[valbok]

https://github.com/valbok/QtAVPlayer/pull/309 this potentially might fix it

[valbok]

https://github.com/valbok/QtAVPlayer/pull/316 QAVStream::framesCount() returns expected num of frames, could be used for perf issues.

[valbok]

https://github.com/valbok/QtAVPlayer/pull/317 Please reopen if there is still an issue. Thanks for your support.