search

jjm2473 / ffmpeg-rk

支持rockchip mpp硬件加速编解码的ffmpeg版本。Forked ffmpeg that supports rkmpp decodeing, encoding and rga scaling
https://ffmpeg.org
Other
178 stars 31 forks source link

How to verify hardware transcoding via VAAPI jellyfin #25

Closed chronicblondiee closed 10 months ago

chronicblondiee commented 12 months ago

Hi,

Sorry not sure where this should go but wanted to ask the maintainer for some advice thank you for your work on supporting this SoC

I am testing your jellyfin image on orangpi-5-b-plus rk3588 SoC ( https://hub.docker.com/r/jjm2473/jellyfin-mpp)

hardware drivers following armbian instructions PPA install https://www.armbian.com/orangepi-5/

I am not very familiar with this hardware so I am curious how you can verify if hardware transcoding is working correctly via jellyfin FFMPEG

I can confirm that transcoding is happening and performance is good but not exactly sure how to verify the device in use to that is preforming the encoding.

I also converted your docker run command into a docker-compose file for ease of use note i just took the loop and add them as a list of devices in the environment variables. 

# confirmed devices on system
/dev/rga
/dev/dri
/dev/dma_heap
version: '3.3'
services:
    jellyfin-mpp:
        image: jjm2473/jellyfin-mpp:latest
        container_name: jellyfin
        privileged: true # needed to work with hw transcoding
        restart: unless-stopped
        environment:
          - PUID=1000
          - PGID=1000
          - TZ=Etc/UTC
          - JELLYFIN_PublishedServerUrl=192.168.1.181 #optional
          - /dev/iep
          - /dev/rga
          - /dev/dri
          - /dev/dma_heap
          - /dev/mpp_service
          - /dev/mpp-service
          - /dev/vpu_service
          - /dev/vpu-service
          - /dev/hevc_service
          - /dev/hevc-service
          - /dev/rkvdec
          - /dev/rkvenc
          - /dev/avsd
          - /dev/vepu
          - /dev/h265e
        ports:
          - 8096:8096
          - 8920:8920 #optional
          - 7359:7359/udp #optional
          - 1900:1900/udp #optional
        volumes:
            - /home/jelly/jellyfin-rk/config:/config
            - /mnt:/mnt

I am hoping that you can give some advice on the best way to verify I have set hardware acceleration using VAAPI using device /dev/dri/renderD128

Thanks in advance

danielpinto8zz6 commented 10 months ago

Hi @chronicblondiee , have you been able to check whether is working correctly?

chronicblondiee commented 10 months ago

@danielpinto8zz6 I was able to verify that that the HW transcoding was happening by looking at the cmd output of the process it created and the transcoding logs. Have you been able to get hardware transcoding working on your device ?

I have now gone back to reproduce it and it seems that hw transcoding is no longer working to my clients failing for both h264 media and hevc/h265.

I used various 4K HEVC / H265 files on different clients ( Web/ Android/ TV ) but everytime it trys to hardware transcode playback now fails...

trying to force the clients to play transcoded media results in playback error now ... not exactly sure whats changed but you can verify your transcoding is working in 2 ways.

htop or other process monitor and look for the ffmpeg calls you should see the rk_mpp in the cmd args

you can also check the logs in your apps "./config/log/" this will show the log for all the transcodes, direct plays and media remuxs that happen.

FFmpeg.DirectStream-2023-10-18_14-44-50_db5a399ad87ca4b075ac4036195ae655_6f80530d.log  FFmpeg.Transcode-2023-10-18_14-37-53_db5a399ad87ca4b075ac4036195ae655_46cf2a4f.log
FFmpeg.Remux-2023-10-18_14-49-29_ceabd6fff30398bb2c12fad087aac82b_498f92a9.log

Sample of what transcoding looks like for me when i try to transcode HEVC to H264

Shows the stream mapping for the hardware

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> h264 (h264_rkmpp))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[hevc_rkmpp @ 0x2ed83c60] Decoder noticed an info change (2560x1440), stride(2816x1440), format=0
[h264_rkmpp @ 0x2e8e9550] Unsupport profile -99, force set to 100
[h264_rkmpp @ 0x2e8e9550] Unsupport level -99, force set to 51

full log

/videos/5021fbb4-6f52-9b0e-20af-6b4a6ab07fd9/hls1/main/0.ts

{"Protocol":0,"Id":"5021fbb46f529b0e20af6b4a6ab07fd9","Path":"/mnt/shared/game-clips/Escape from Tarkov_replay_2023.08.17-22.31.mp4","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"mov,mp4,m4a,3gp,3g2,mj2","Size":11326700544,"Name":"Escape from Tarkov_replay_2023.08.17-22.31","IsRemote":false,"ETag":"a1990426c56bc2e5c2ec4ae0613469b4","RunTimeTicks":18000259072,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"hevc","CodecTag":"hvc1","Language":"und","ColorRange":null,"ColorSpace":"bt709","ColorTransfer":"bt709","ColorPrimaries":"reserved","DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/15360","CodecTimeBase":null,"Title":null,"VideoRange":"SDR","VideoRangeType":"SDR","VideoDoViTitle":null,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"DisplayTitle":"1080p HEVC SDR","NalLengthSize":null,"IsInterlaced":false,"IsAVC":null,"ChannelLayout":null,"BitRate":50006651,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":true,"IsForced":false,"Height":1440,"Width":2560,"AverageFrameRate":60.000813,"RealFrameRate":60,"Profile":"Main","Type":1,"AspectRatio":"16:9","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p","Level":150,"IsAnamorphic":null},{"Codec":"aac","CodecTag":"mp4a","Language":"und","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/48000","CodecTimeBase":null,"Title":null,"VideoRange":null,"VideoRangeType":null,"VideoDoViTitle":null,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"DisplayTitle":"AAC - Stereo - Default","NalLengthSize":null,"IsInterlaced":false,"IsAVC":null,"ChannelLayout":"stereo","BitRate":319752,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":48000,"IsDefault":true,"IsForced":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"Profile":"LC","Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null}],"MediaAttachments":[],"Formats":[],"Bitrate":50340166,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":null,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null}

/usr/bin/ffmpeg -analyzeduration 200M  -i file:"/mnt/shared/game-clips/Escape from Tarkov_replay_2023.08.17-22.31.mp4" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 libx264 -preset veryfast -crf 23 -maxrate 50006651 -bufsize 100013302 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -sc_threshold:v:0 0 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*a)\,min(2560\,1440*a))/2)*2:trunc(min(max(iw/a\,ih)\,min(2560/a\,1440))/2)*2,format=yuv420p" -codec:a:0 copy -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/config/transcodes/3b8f3477565678060b6ef7c40e4b1267%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/config/transcodes/3b8f3477565678060b6ef7c40e4b1267.m3u8"

ffmpeg "-analyzeduration" "10000000" "-probesize" "10000000" "-i" "file:/mnt/shared/game-clips/Escape from Tarkov_replay_2023.08.17-22.31.mp4" "-map_metadata" "-1" "-map_chapters" "-1" "-threads" "0" "-map" "0:0" "-map" "0:1" "-map" "-0:s" "-vf" "scale_rga=1920x1080" "-codec:v:0" "h264_rkmpp" "-flags:v" "-global_header" "-b:v" "49366651" "-maxrate" "50006651" "-bufsize" "100013302" "-force_key_frames:0" "expr:gte(t,0+n_forced*3)" "-sc_threshold:v:0" "0" "-codec:a:0" "copy" "-copyts" "-avoid_negative_ts" "disabled" "-max_muxing_queue_size" "2048" "-f" "hls" "-max_delay" "5000000" "-hls_time" "3" "-hls_segment_type" "mpegts" "-start_number" "0" "-hls_segment_filename" "/config/transcodes/3b8f3477565678060b6ef7c40e4b1267%d.ts" "-hls_playlist_type" "vod" "-hls_list_size" "0" "-y" "/config/transcodes/3b8f3477565678060b6ef7c40e4b1267.m3u8"
ffmpeg version git-2022-06-18-83b0e20 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 8.3.0 (OpenWrt GCC 8.3.0 r0-83b0e20)
  configuration: --enable-cross-compile --cross-prefix=aarch64-openwrt-linux-gnu- --arch=aarch64 --cpu=cortex-a53 --target-os=linux --prefix=/usr --pkg-config=pkg-config --enable-shared --enable-static --enable-pthreads --enable-zlib --disable-doc --disable-debug --disable-lzma --disable-vaapi --disable-vdpau --disable-outdevs --disable-runtime-cpudetect --enable-lto --enable-neon --enable-vfp --enable-gnutls --disable-encoder=svq1 --enable-libopus --enable-hardcoded-tables --enable-libmp3lame --enable-gpl --enable-libx264 --enable-version3 --enable-nonfree --enable-libdrm --enable-librga --enable-rkmpp --enable-libfdk-aac
  libavutil      57. 33.101 / 57. 33.101
  libavcodec     59. 42.101 / 59. 42.101
  libavformat    59. 30.100 / 59. 30.100
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 46.103 /  8. 46.103
  libswscale      6.  8.102 /  6.  8.102
  libswresample   4.  8.100 /  4.  8.100
  libpostproc    56.  7.100 / 56.  7.100
[hevc @ 0x2e78f600] VPS 0 does not exist
[hevc @ 0x2e78f600] SPS 0 does not exist.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:/mnt/shared/game-clips/Escape from Tarkov_replay_2023.08.17-22.31.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf58.76.100
  Duration: 00:30:00.03, start: 0.000000, bitrate: 50340 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709/reserved/bt709), 2560x1440 [SAR 1:1 DAR 16:9], 50006 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Codec AVOption sc_threshold (Scene change threshold) specified for output file #0 (/config/transcodes/3b8f3477565678060b6ef7c40e4b1267.m3u8) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> h264 (h264_rkmpp))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[hevc_rkmpp @ 0x2ed83c60] Decoder noticed an info change (2560x1440), stride(2816x1440), format=0
[h264_rkmpp @ 0x2e8e9550] Unsupport profile -99, force set to 100
[h264_rkmpp @ 0x2e8e9550] Unsupport level -99, force set to 51
Output #0, hls, to '/config/transcodes/3b8f3477565678060b6ef7c40e4b1267.m3u8':
  Metadata:
    encoder         : Lavf59.30.100
  Stream #0:0: Video: h264, drm_prime(tv, unknown/reserved/reserved, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 49366 kb/s, 60 fps, 90k tbn (default)
    Metadata:
      encoder         : Lavc59.42.101 h264_rkmpp
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default)
frame=    4 fps=0.0 q=-0.0 size=N/A time=-00:00:00.01 bitrate=N/A dup=0 drop=1 speed=N/A    
frame=   61 fps=0.0 q=-0.0 size=N/A time=00:00:00.99 bitrate=N/A dup=0 drop=1 speed=1.96x    
frame=  128 fps=122 q=-0.0 size=N/A time=00:00:02.05 bitrate=N/A dup=0 drop=1 speed=1.97x    
[hls @ 0x2e9a6da0] Opening '/config/transcodes/3b8f3477565678060b6ef7c40e4b12670.ts' for writing
frame=  189 fps=121 q=-0.0 size=N/A time=00:00:03.08 bitrate=N/A dup=0 drop=1 speed=1.97x    
frame=  254 fps=121 q=-0.0 size=N/A time=00:00:04.14 bitrate=N/A dup=0 drop=1 speed=1.98x    
frame=  318 fps=121 q=-0.0 size=N/A time=00:00:05.21 bitrate=N/A dup=0 drop=1 speed=1.99x    
[hls @ 0x2e9a6da0] Opening '/config/transcodes/3b8f3477565678060b6ef7c40e4b12671.ts' for writing
frame=  379 fps=121 q=-0.0 size=N/A time=00:00:06.24 bitrate=N/A dup=0 drop=1 speed=1.99x    

[q] command received. Exiting.

[hls @ 0x2e9a6da0] Opening '/config/transcodes/3b8f3477565678060b6ef7c40e4b12672.ts' for writing
frame=  403 fps=121 q=-0.0 Lsize=N/A time=00:00:06.62 bitrate=N/A dup=0 drop=1 speed=1.99x    
video:19935kB audio:282kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
jjm2473 commented 10 months ago

Jellyfin logs start with FFmpeg.Transcode-* means transcoding.

I don't think Rockchip will support VAAPI.

jjm2473 commented 10 months ago

@chronicblondiee From your compose file:

version: '3.3'
services:
    jellyfin-mpp:
        image: jjm2473/jellyfin-mpp:latest
        container_name: jellyfin
        privileged: true # needed to work with hw transcoding
        restart: unless-stopped
        environment:
          - PUID=1000
          - PGID=1000
          - TZ=Etc/UTC
          - JELLYFIN_PublishedServerUrl=192.168.1.181 #optional
          - /dev/iep
          - /dev/rga
          - /dev/dri
          - /dev/dma_heap
          - /dev/mpp_service
          - /dev/mpp-service
          - /dev/vpu_service
          - /dev/vpu-service
          - /dev/hevc_service
          - /dev/hevc-service
          - /dev/rkvdec
          - /dev/rkvenc
          - /dev/avsd
          - /dev/vepu
          - /dev/h265e
        ports:
          - 8096:8096
          - 8920:8920 #optional
          - 7359:7359/udp #optional
          - 1900:1900/udp #optional
        volumes:
            - /home/jelly/jellyfin-rk/config:/config
            - /mnt:/mnt

There is some errors: /dev/rga, etc. should not in envrionment, should be --device.

Just run this command on your board and check output:

echo docker run --name jellyfin \
    --privileged \
    --restart=unless-stopped -td \
    `for dev in iep rga dri dma_heap mpp_service mpp-service vpu_service vpu-service \
        hevc_service hevc-service rkvdec rkvenc avsd vepu h265e ; do \
      [ -e "/dev/$dev" ] && echo " --device /dev/$dev"; \
    done` \
    --dns=172.17.0.1 \
    -p 8096:8096 \
    -v /root/jellyfin/config:/config \
    -v /mnt:/mnt \
    jjm2473/jellyfin-mpp:latest