Not an issue but an improved version for raspberry pi

[palazzoni]

I am interested in using OpenMax library to hardware decode on pi zero: I have modified decoder.c code to decode stream from socket with broadcom ilclient but I am not an expert c/c++ coder so I need help to review the code. I attach the modified cose here, you can simply put a switch at compile time to compile one of the ffmpeg or ilclient/openmax decoder.c code. Please give me a feedback if any of you are interested in this, I am working on project to made a pi zero android docking station for desktop use. decoder.c.log

[rom1v]

Thank you for sharing your work.

But as preliminary question, isn't OMX already supported by FFmpeg? If it is the case, I guess scrcpy should work with a FFmpeg compiled with OMX support (not tested).

[palazzoni]

Hi, FFmpeg claim to support raspberry hardware decode/encode with omx but from my test and from feedback from various users it does not offer great performance, expecially on pi zero: ffmpeg use h264_mmal/h264_omx but it’s buggy and this results in unusable video performance. I test it scrcpy server directly with omx player and it working like a sharm, perfectly from video perspective so for this reason I want to add direct support for decoding. Regards,

Palazzoni Michele

Il giorno 23 set 2018, alle ore 18:48, Romain Vimont notifications@github.com ha scritto:

Thank you for sharing your work.

But as preliminary question, isn't OMX already supported by FFmpeg? If it is the case, I guess scrcpy should work with a FFmpeg compiled with OMX support (not tested).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[rom1v]

FFmpeg claim to support raspberry hardware decode/encode with omx but from my test and from feedback from various users it does not offer great performance, expecially on pi zero: ffmpeg use h264_mmal/h264_omx but it’s buggy and this results in unusable video performance.

Do you know why it does not use the same API you use here?

If what you say is correct, it could be worth it to provide a raspberry-specific implementation. However, it requires some work to integrate it properly, so that it can be enabled with a simple compilation flag (so that we can compile with something like meson buildir -Duse_omx=true).

[palazzoni]

I’ll investigate more on ffmpeg implementation of omx api and let you know...but I think that for me is more simple to compile scrcpy directly with omx api instead of correcting ffmpeg implementation (thanks for suggestion). I’ll let you know as soon as I have time to do so. Thank you very much for your response. Best regards,

Palazzoni Michele

Il giorno 24 set 2018, alle ore 21:21, Romain Vimont notifications@github.com ha scritto:

FFmpeg claim to support raspberry hardware decode/encode with omx but from my test and from feedback from various users it does not offer great performance, expecially on pi zero: ffmpeg use h264_mmal/h264_omx but it’s buggy and this results in unusable video performance.

Do you know why it does not use the same API you use here?

If what you say is correct, it could be worth it to provide a raspberry-specific implementation. However, it requires some work to integrate it properly, so that it can be enabled with a simple compilation flag (so that we can compile with meson buildir -Duse_omx=true).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[palazzoni]

Compiled with omx ilclient video decoder and decoded frame passed to sdl ffmpeg for screen rendering ( corrections done on decoder.c) but don’t have desidered performance (2-3 seconds delay), no way...I must wait for ffmpeg hardware optimizations on pi zero.

[palazzoni]

I try to implement also rendering with ilclient omx library but then it is difficult to render on sdl Window, no way to have visible mouse pointer (also problems on layers in window), no way out...

[RpiDude]

Compiled with omx ilclient video decoder and decoded frame passed to sdl ffmpeg for screen rendering ( corrections done on decoder.c) but don’t have desidered performance (2-3 seconds delay), no way...I must wait for ffmpeg hardware optimizations on pi zero.

Hello is there any chance you could share this compiled package, I don't care if it lags, I just need to show static images from my android device on the pi.

[palazzoni]

Hi, I upload scrcpyomx binary on https://github.com/palazzoni/scrcpyomx , this binary was compiled with decoder.c.log file attached on this thread.

[rom1v]

@palazzoni Why don't you just fork and apply your changes, so that you share the source code?

[palazzoni]

Only because I change decoder.c file in all source tree :-)

[rom1v]

Only because I change decoder.c file in all source tree :-)

It's still simpler if you just add a commit with your changes in your own fork.

[Koloses]

@palazzoni Im having issues trying to compile this on top of audio branch on Pi zero. Could you kindly explain flags/dependency changes needed in build system for it to work? That would help a lot. So far I added dummy .pk files in pkgconfig and deps in meson to point to Pi includes for the hw codec and added -std=c99 flag for compiling decoder.c but that is not enough and I still get errors so I must've done something wrong. I need this to turn pi zero into hdmi dongle for android phone that doesn't have hdmi nor mhl support.

[mufunyo]

I've been looking into getting scrcpy to work on a Raspberry Pi 3+, and so far I haven't been able to get any better performance than slideshow at anything 480p or bigger, and about 30fps at 240p (no kidding). @palazzoni 's scrcpyomx doesn't seem to do anything that regular scrcpy doesn't do - that is it also uses software decoding with the same abysmal performance.

I tried compiling ffmpeg with mmal decode support, and modified stream.c to call avcodec_find_decoder_by_name("h264_mmal") instead of avcodec_find_decoder(AV_CODEC_H264), but I get the following log output: ERROR: Could not send video packet: -1313558101 Curiously it didn't log "H.264 decoder not found" or "Could not open decoder" as you might expect if I did something wrong with regards to attempting to make it use mmal.

@rom1v , would you be interested in taking a look if I shipped you a complete RasPi3+ kit or donated enough for you to order one yourself? I'm not familiar with this codebase, nor with ffmpeg, so I'm pretty lost in the woods.

[rom1v]

@mufunyo

would you be interested in taking a look if I shipped you a complete RasPi3+ kit or donated enough for you to order one yourself?

Sorry, I can't guarantee I'll have time to investigate this. Anyway, I'm quite clueless about how to solve it.

Did you try: https://github.com/legotheboss/YouTube-files/wiki/(RPi)-Compile-FFmpeg-with-the-OpenMAX-H.264-GPU-acceleration ?

[mufunyo]

Did you try: https://github.com/legotheboss/YouTube-files/wiki/(RPi)-Compile-FFmpeg-with-the-OpenMAX-H.264-GPU-acceleration ?

Yes, although the rabbit hole goes deeper than that. That guide only enables OMX (OpenMAX) support in ffmpeg, which is limited to encoding only. To get hardware decode on Broadcom, you need to enable the MMAL decoder. Also, apparently MMAL decoding (possibly) broke some time ago, so for highest chances of success I used ffmpeg version 3.1.10: https://raspberrypi.stackexchange.com/questions/66923/rpi-3-performance-issue-on-ffmpeg-with-mmal-support

Unfortunately I haven't been able to test ffmpeg with hardware decode directly (instead of scrcpy linking to libavcodec) because gcc throws an error while linking ffplay, so I've had to compile with --disable-ffplay.

I also wanted to look into touch screen support (which involves SDL which I am pretty familiar with) but I can't even try that if scrcpy barely runs at all to begin with.

[reversegear]

Did you try: https://github.com/legotheboss/YouTube-files/wiki/(RPi)-Compile-FFmpeg-with-the-OpenMAX-H.264-GPU-acceleration ?

Yes, although the rabbit hole goes deeper than that. That guide only enables OMX (OpenMAX) support in ffmpeg, which is limited to encoding only. To get hardware decode on Broadcom, you need to enable the MMAL decoder. Also, apparently MMAL decoding (possibly) broke some time ago, so for highest chances of success I used ffmpeg version 3.1.10: https://raspberrypi.stackexchange.com/questions/66923/rpi-3-performance-issue-on-ffmpeg-with-mmal-support

Unfortunately I haven't been able to test ffmpeg with hardware decode directly (instead of scrcpy linking to libavcodec) because gcc throws an error while linking ffplay, so I've had to compile with --disable-ffplay.

I also wanted to look into touch screen support (which involves SDL which I am pretty familiar with) but I can't even try that if scrcpy barely runs at all to begin with.

any success with the hardware acceleration

[mufunyo]

any success with the hardware acceleration

No. If I did, I would've posted about it here. For now I defer to someone with more Raspi and/or ffmpeg knowledge.

[reversegear]

@mufunyo can this project help in anyway to get rpi support. https://github.com/homeworkc/lazycast

[mufunyo]

@mufunyo can this project help in anyway to get rpi support. https://github.com/homeworkc/lazycast

That looks to be linking OMX IL directly, which would require rewriting parts of scrcpy's decoding chain. I'd much rather let ffmpeg do all the lifting if at all possible, for simplicity of code and to prevent too much platform-specific code entering the repo.

[mufunyo]

I did some more testing and found out a bunch of things. I used ffplay to test with, because it also draws to SDL like scrcpy, but it is a "known working" app and it has some verbose console logging.

With a default config.txt, gpu_mem is too low so h264_mmal can't allocate frames to decode to and nothing works. So first I had to set gpu_mem to 512.

Then, I got about 4 fps decoding for 720p60 content, 2 fps decoding for 1080p60 content, and I guess 6-ish fps decoding for 1080p24 content. I mean, it's better than the seconds per frame that I gpt when I tried to decode anything on the CPU, but it's not exactly real time.

I was using an underpowered power supply (the highest ampere rated phone charger I have in the house that isn't USB-PD or QC, neither of which are supported by the Pi anyway), so I decided to move to something beefier: an ATX power supply rated to 20A on its 5V output. Still the exact same decoding performance, but hey, no more undervoltage warnings, and I got rid of that pesky lightning bolt icon.

The next logical step was to overclock the Pi, so I glued an oversized heatsink that just barely fit between the camera and display connectors onto the SoC with some heatsink plaster and set force_turbo to 1, arm_freq to 1000 and gpu_freq to 500.

Still the exact same abysmal video decoding performance in ffplay.

...

As much as I'd like to see an appliance-ified version of scrcpy that doesn't require a full-blown x86 PC, I'm not sure how suited the Raspberry Pi is as a hardware platform. The LibreELEC guys sure seem to have made it work, but that can barely run anything other than Kodi, and scrcpy isn't Kodi. It doesn't even include Xorg, so before anything, scrcpy would need to be modified to render using KMS/DRM.

This is all way above my head, so I'm giving up. If anyone wants to try to make it work, please do and report back in this thread.

[ghost]

Ive tried this app with prebuilt server (maybe the binary its not crossplatform?) and it doesnt run at all. Ive tried with my own android-sdk too, but so far I couldnt (but I think they are just tools not a full sdk)

pi@raspberrypi:~ $ scrcpy 
INFO: scrcpy 1.10 <https://github.com/Genymobile/scrcpy>
/usr/local/share/scrcpy/scrcpy-server.jar: 1 file pushed. 3.0 MB/s (22662 bytes in 0.007s)
error: closed
ERROR: "adb reverse" returned with value 1
WARN: 'adb reverse' failed, fallback to 'adb forward'

[rom1v]

@Askmewho Does your device have at least Android 5?

[ghost]

Responded on newer issue. Thanks