search

mickelson / attract

A graphical front-end for command line emulators that hides the underlying operating system and is intended to be controlled with a joystick or gamepad.
http://attractmode.org
GNU General Public License v3.0
397 stars 115 forks source link

There is no HW decoding anymore for MPEG4 and H263 on RPI4 through mmal only for H264 and i think HEVC #625

Closed joyrider3774 closed 4 years ago

joyrider3774 commented 4 years ago

source from rpi engineer https://www.raspberrypi.org/forums/viewtopic.php?t=268356#p1628856

I don't know if it ever will get such support again.

Relevant part:

Pi0-3 have hardware accelerated decode for H264, MPEG4, H263, and through optional codec licences for MPEG2 and VC1. Pi4 has the same hardware accelerated decode for H264, but not the other codecs. It also has a separate block for HEVC.

trying to play MPEG4 video through mmal produces following errors in attract

mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x1d50250 (EINVAL)
mmal: mmal_port_disable: port vc.ril.video_decode:in:0(MP4V)(0x1d50250) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:out:0(0x1d50560) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:ctr:0(0x1d4ff30) is not enabled
Could not open video decoder for file: /home/pi/.attract/menu-art/snap/All Displays.mp4
pi@retropieTV:~/develop/ffmpeg $ ffprobe "/home/pi/.attract/menu-art/snap/All Displays.mp4"
ffprobe version N-98041-ge34686d7ac Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 8 (Raspbian 8.3.0-6+rpi1)
  configuration: --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-libass --enable-libbs2b --enable-libcaca --enable-libcodec2 --enable-libx265 --enable-libxvid --enable-omx --enable-openal --enable-opengl --enable-omx-rpi --enable-mmal --enable-neon --enable-libdrm --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc
  libavutil      56. 49.100 / 56. 49.100
  libavcodec     58. 90.100 / 58. 90.100
  libavformat    58. 44.100 / 58. 44.100
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 84.100 /  7. 84.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, avi, from '/home/pi/.attract/menu-art/snap/All Displays.mp4':
  Metadata:
    date            : 2018-01-28T18:10:03.00012
    encoder         : Lavf56.19.100
  Duration: 00:00:31.07, start: 0.000000, bitrate: 2857 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 854x480 [SAR 1:1 DAR 427:240], 2715 kb/s, SAR 1280:1281 DAR 16:9, 30 fps, 30 tbr, 30 tbn, 30 tbc
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 128 kb/s
pi@retropieTV:~/develop/ffmpeg $

Maybe attract should not try to use mmal for MPEG4 anymore on a RPI4 but use software decoder ? Videos using H264 do still play fine using mmal. Perhaps should try adding support for HEVC but not sure if libavcodec (ffmpeg) supports it for mmal or if needs some other implementation, i think it's still being worked on rpi side itselve

2play commented 4 years ago

Instead of going crazy and since using mmal I took the advise of @joyrider3774 and re encoding my mp4 with libx264 I have tested Attract setup mp4 and is working great and smaller size.

Example if anyone interested (of course you can use your paths etc) sudo ffmpeg -i /home/pi/.attract/menu-art/snap/Attract\ Mode\ Setup.mp4 -vcodec libx264 -acodec aac /home/pi/.attract/menu-art/snap/new//Attract\ Mode\ Setup.mp4

joyrider3774 commented 4 years ago

@2play best to provide also -pix_fmt yuv420p mmal H264 decoder only supports that pixelformat. your convert line won't work if videos are for example yuv444p instead of yuv420p. For hyperpie video's your command works because they already are yuv420p but for other videos (game snaps) in other pixelformats it won't work. I also made pr that will revert to software mode when playing those types of video's not supported on RPI4 through mmal anymore (only h264 is supported now through mmal on rpi4, on all other pi's the other codecs are still supported)

also just to confirm when playing through ffplay on rpi4 you get same mmal errors

 ffplay -codec:v mpeg4_mmal ~/.attract/menu-art/snap/All\ Displays.mp4
[mpeg4 @ 0xa7801870] Failed to parse extradata0KB sq=    0B f=0/0
Input #0, avi, from '/home/pi/.attract/menu-art/snap/All Displays.mp4':
  Metadata:
    date            : 2018-01-28T18:10:03.00012
    encoder         : Lavf56.19.100
  Duration: 00:00:31.07, start: 0.000000, bitrate: 2857 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 854x480 [SAR 1:1 DAR 427:240], 2715 kb/s, SAR 1280:1281 DAR 16:9, 30 fps, 30 tbr, 30 tbn, 30 tbc
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 128 kb/s
mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0xa78275e0 (EINVAL)
mmal: mmal_port_disable: port vc.ril.video_decode:in:0(MP4V)(0xa78275e0) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:out:0(0xa7827900) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:ctr:0(0xa78272d0) is not enabled
  37.28 M-A: -0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0
2play commented 4 years ago

tx and noted @joyrider3774 Yes I wanted to initially fix the 205+ videos on HyperPie Theme. So for that we are good yes.

Ill run and test your above once my conversion completes

Would you care sharing your revert to software pr that your mentioned above?

joyrider3774 commented 4 years ago

check the code in the pr (pull request) https://github.com/mickelson/attract/pull/626/commits/98bf92fbf3f685b29974dfee136197f5862680f6 when that code is applied with you just recompile attract and add PLATFORM_RPI4=1 to make line along with the other variables (use_mmal and use_drm)

also in windows i created batch files to convert all video's in one commnand using ffmpeg, i can chat them if needed

2play commented 4 years ago

I did a quick test on what you mentioned:

Orig file

[mpeg4 @ 0xa7801870] Failed to parse extradata0KB sq=    0B f=0/0
Input #0, avi, from '/home/pi/.attract/menu-art/snap/All Displays.mp4':
  Metadata:
    date            : 2018-01-28T18:10:03.00012
    encoder         : Lavf56.19.100
  Duration: 00:00:31.07, start: 0.000000, bitrate: 2857 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 854x480 [SAR 1:1 DAR 427:240], 2715 kb/s, SAR 1280:1281 DAR 16:9, 30 fps, 30 tbr, 30 tbn, 30 tbc
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 128 kb/s
mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0xa7827680 (EINVAL)
mmal: mmal_port_disable: port vc.ril.video_decode:in:0(MP4V)(0xa7827680) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:out:0(0xa7827990) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:ctr:0(0xa7827360) is not enabled

Encoded File

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/pi/.attract/menu-art/snap/new/All Displays.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    date            : 2018-01-28T18:10:03.00012
    encoder         : Lavf58.20.100
  Duration: 00:00:31.08, start: 0.000000, bitrate: 1705 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 854x480 [SAR 1280:1281 DAR 16:9], 1568 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 129 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0xa7939b00 (EINVAL)
mmal: mmal_port_disable: port vc.ril.video_decode:in:0(MP4V)(0xa7939b00) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:out:0(0xa7939e10) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:ctr:0(0xa79397e0) is not enabled
2play commented 4 years ago

check the code in the pr (pull request) 98bf92f when that code is applied with you just recompile attract and add PLATFORM_RPI4=1 to make line along with the other variables (use_mmal and use_drm)

also in windows i created batch files to convert all video's in one commnand using ffmpeg, i can chat them if needed

@joyrider3774 Just want to say thank you for your contribution and work.

I use this on Pi4. Added the -pix_fmt yuv420p as you advised. Can you please confirm?

cd /home/pi/.attract/menu-art/snap/
for i in *.mp4;
  do name=`echo "$i" | cut -d'.' -f1`
  echo "$name"
  sudo ffmpeg -i "$i" -vcodec libx264 -acodec aac -pix_fmt yuv420p ./new/"${name}.mp4"
done
joyrider3774 commented 4 years ago

your second command is wrong in testing the video you need to use h264_mmal decoder like so when trying to play h264 with mmal in ffplay (if the encoded video is h264)

ffplay -codec:v h264_mmal -loop 0 ~/.attract/menu-art/snap/new/Attract\ Mode\ Setup.mp4

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/pi/.attract/menu-art/snap/new/Attract Mode Setup.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    date            : 2018-01-31T10:01:11.00494
    encoder         : Lavf58.20.100
  Duration: 00:00:26.07, start: 0.000000, bitrate: 1114 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 854x480 [SAR 1280:1281 DAR 16:9], 979 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
  11.72 A-V:  0.001 fd=   5 aq=   31KB vq=  139KB sq=    0B f=0/0

also it's possible to just copy the audio stream and only converting video but not sure you want that, it's ok to recompress audio as well i just would set -f MP4 exlicitly also

ffmpeg -i /home/pi/.attract/menu-art/snap/Attract\ Mode\ Setup.mp4 -c:v libx264 -c:a copy -pix_fmt yuv420p -f MP4 /home/pi/.attract/menu-art/snap/new//Attract\ Mode\ Setup.mp4
2play commented 4 years ago

yes confirm above ffplay -codec:v h264_mmal is ok!

joyrider3774 commented 4 years ago

@2play my latest pr will detect the errors when playing with mmal and revert to software decoding if an error occurred, when doing so it will write entries to the log saying the hardware decoding is not supported / working. no PLATFORM_RPI flag is needed anymore and should work on any system supporting mmal where USE_MMAL=1 is provided see code here https://github.com/mickelson/attract/pull/627/commits/60008f08634067a8f6fb978974a04603ebc595a2

2play commented 4 years ago

Tx I'll update!

joyrider3774 commented 4 years ago

it's not in master yet though only submitted a pr