Feature request: enable/integrate rockchip hwaccel support

[dev-zero]

It would be really nice if the jellyfin-ffmpeg would support the rockchip hardware acceleration.

For 4.1 there's a maintained ffmpeg-fork here with additional patches it seems: https://github.com/rockchip-linux/ffmpeg/tree/rockchip/release/4.1 but basic rockchip acceleration support was already included via the RK-MPP in ffmpeg-3.

This library is needed to enable support: https://github.com/rockchip-linux/mpp

Communication with the kernel is then through a /dev/videoN device if I understood it correctly.

[JustAMan]

We'd like to keep in line with upstream ffmpeg as much as possible. Do you know if ffmpeg 4.2 or newer has this enabled?

[dev-zero]

It seems that support for rockchip-acceleration was added in ffmpeg-3 (and continues to be there). It is by default not enabled since it needs the mpp library:

 ~/Downloads/ffmpeg-4.2.2 $ ./configure --help | grep mpp
  --enable-rkmpp           enable Rockchip Media Process Platform code [no]

[JustAMan]

Looking at their repo I don't see a license. We cannot redistribute builds without compatible licenses... please investigate the licensing of their code and report back :)

[dev-zero]

The debian control file and the headers say Apache 2.0 (which is probably why ffmpeg requires to be built with --enable-version3):

• https://github.com/rockchip-linux/mpp/blob/develop/debian/copyright
• https://github.com/rockchip-linux/mpp/blob/develop/mpp/inc/mpp.h

Just did a succesful buidl test with the MPP library from their devel branch (the latest release is quiet old) and ffmpeg with ./configure --enable-rkmpp --enable-version3 --enable-libdrm.

[JustAMan]

I think you can try and make a PR yourself, this is probably the file you need to modify: https://github.com/jellyfin/jellyfin-ffmpeg/blob/jellyfin/docker-build.sh

[RlndVt]

I am also interested in having rockchip hwaccel, specifically for the Helios64. I've noticed that the mpp repo https://github.com/rockchip-linux/mpp doesn't exist anymore. There is a alternative here https://github.com/HermanChen/mpp. Don't know how official it is.

Is this a problem for redistribution?

[nyanmisaka]

I am also interested in having rockchip hwaccel, specifically for the Helios64. I've noticed that the mpp repo https://github.com/rockchip-linux/mpp doesn't exist anymore. There is a alternative here https://github.com/HermanChen/mpp. Don't know how official it is.

Is this a problem for redistribution?

I think it's still official. It has been moved to a personal repo of a Rockchip employee. There are only decoders for rkmpp in ffmpeg, encoders and scaler have not been implemented.

[NablaMetal]

Any news on this one ? I am also very insterested in having hwaccel enabled for jellyfin upstream.

[dev-zero]

Not from me unfortunatley: I moved from a Rockchip back to Intel

[tandy-1000]

Does anyone know what actually needs to be done to enable this?

If I understand correctly:

• [ ] Package rockchip-mpp in the Jellyfin repo
• [ ] Add it to the requirements arm64 build and enable the build flag

[nyanmisaka]

You are free to do it. But note that ffmpeg only has rockchip’s hardware decoder, which means transcoding speed will still bottlenecked by software scaling and software encoding.

[NablaMetal]

@nyanmisaka absolutely. That is why I personnaly moved to x86 platform whith much much better performance.

[tandy-1000]

You are free to do it. But note that ffmpeg only has rockchip’s hardware decoder, which means transcoding speed will still bottlenecked by software scaling and software encoding.

That makes sense, I think this will have a reasonable benefit on rockchip servers.

Does my todo list make sense? If so, how would I get started with packaging rockchip-mpp in the jellyfin repo?

[nyanmisaka]

Sure, make sense. You can add related build configs in: https://github.com/jellyfin/jellyfin-ffmpeg/blob/jellyfin/debian/rules https://github.com/jellyfin/jellyfin-ffmpeg/blob/jellyfin/docker-build.sh

[prahal]

At least, the Armbian edge kernel for helios64 has hantro-vpu encoder/decoder and rockchip_vdec support. FFmpeg support is not official but active upstream seems to currently be https://github.com/jernejsk/FFmpeg (devel in vc1-v2 branch) shipped by LibreELEC in https://github.com/LibreELEC/LibreELEC.tv/tree/master/packages/multimedia/ffmpeg/patches/v4l2-request. Maybe https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-drmprime/ffmpeg-001-v4l2-drmprime.patch is of use too, but I do not know if required. I have not yet tested the above, but at first glance, a few patches are not required for non-Kodi FFmpeg. I have an helios64 with the Jellyfin docker container so I will be eager to test.

[prahal]

Another option might be the vaapi v4l2-request backend https://github.com/bootlin/libva-v4l2-request (this repository is inactive but there are newer low activity forks)

[RicardoCst]

I think this is a good idea because it moves away from the typical: "Oh you need an intel/amd processor with integrated graphics to support transcoding" statement. And its another thing which makes jellyfin different from the rest (emby, plex).

[IngwiePhoenix]

The --enable-rkmpp flag exists in the configure script, but is defaulted to off.

I mainly use the Jellyfin Docker container. Any good way that I can replace that with a self-built jellyfin-ffmpeg that has rkmpp enabled - and then actually use it in Jellyfin itself?

[nyanmisaka]

The --enable-rkmpp flag exists in the configure script, but is defaulted to off.

I mainly use the Jellyfin Docker container. Any good way that I can replace that with a self-built jellyfin-ffmpeg that has rkmpp enabled - and then actually use it in Jellyfin itself?

The fact is that rkmpp hwaccel in ffmpeg is outdated to support rk35xx chips, and it doesn’t support encoding. Nothing would work unless someone rewrite it from scratch.

https://github.com/JeffyCN/FFmpeg/issues/4#issuecomment-1179454405

[jjm2473]

If you just want to use HW accel in jellyfin, my project should be enough https://github.com/jjm2473/ffmpeg-rk/wiki

Note: HW decodeing/encoding works, RGA scale works. No tone mapping supported. Tested on rk3568/rk3588.

Update 2023-11-24: now supports HW mjpeg decoding; rga_scale filter supports setting output pix format, supports yuvj42xp/yuyv422/rgb as input .

[prahal]

@jjm2473 I think mpp kernel driver is a rockchip vendor kernel only thing. Armbian for one has moved to mainline thus as far as I understand I cannot use ffmpeg-rk on an Armbian install.

My understanding is that rockchip has no plan to port mpp drivers upstream. And hantro h264 encoding has been added to the upstream hantro driver. So I believe v4l-request is the way to support upstream kernels.

[jjm2473]

@prahal No, the MPP drivers not upstreamed. But mainline linux can be patch to supports MPP. Not only MPP drivers, but also RGA drivers shoubld be supported, because some SoC has not enough performance to encoding 4K/30FPS video (for example RK3568, up to 1080p/30FPS encoding), must use RGA to scale video.

I have no experience with v4l. The ffmpeg-rk project is only implemented by referring to RK's old ffmpeg and MPP/RGA development documents, so it depends on related kernel functions.

[prahal]

@jjm2473 https://aur.archlinux.org/packages/rockchip-mpp?all_reqs=1 tells tells MPP only works with rockchip BSP kernel ( latest 5.10 in a develop branch).

Where could I get these MPP and RGA drivers against linux 6.4-rcN?

To me, the issue is not that MPP and RGA are not upstream yet. But that rockchip loos like it has no plan to devote developer time to upstream it. This looks like an internal project for their custom kernel that they develop on a public github to help their customers customize products based on their kernel.

[IngwiePhoenix]

linux 6.4-rcN

I hate to quote a meme but... That's the neat part, you don't!

There has not been a single commit to their kernel repo in forever - let alone a new branch to support anything newer than 5.10.x

In fact, here's my current one on a RK3588 (FriendlyElec NanoPi R6s)

# uname -a
Linux FriendlyWrt 5.10.110 #1 SMP Sat Dec 3 01:25:15 CST 2022 aarch64 GNU/Linux

This corresponds to https://github.com/rockchip-linux/kernel/tree/develop-5.10

[prahal]

linux 6.4-rcN

I hate to quote a meme but... That's the neat part, you don't!

Then should never be part of a linux distribution as not security supported. I wonder why Archlinux ships userspace mpp-git https://aur.archlinux.org/packages/mpp-git.

Sadly v4lrequests development is somewhat stalled especially on the vaapi side (there is a ffmpeg v4lrequests specific patch which is more up to date but not upstreamed either). I do not know about scaling with v4lrequests.

[jjm2473]

@prahal Check these drivers and patches: https://github.com/istoreos/istoreos/tree/istoreos-22.03/target/linux/rockchip/rk35xx/files/drivers/video/rockchip https://github.com/istoreos/istoreos/tree/istoreos-22.03/target/linux/rockchip/rk35xx/patches-5.10

Maybe someone can port to linux 6.x?

[prahal]

@jjm2473 if it was that easy I hope rockchip would have paid their developers to do the upstreaming. No definitely MPP is for a rockchip fork of the linux kernel. This is not a Linux kernel, it is only based on it. A little like the Windows TCP/IP stack based on BSD code does not make Windows a BSD kernel, except here the fork is only 3 years old. Like you can add hw video acceleration for a BSD stack to jellyfin ffmpeg you can add the Rockchip kernel video acceleration stack support to jellyfin ffmpeg. This will not support for Linux on Rockchip hardware.

I am after an hw acceleration stack for Armbian current and edge, that is Linux per see. So MPP is of no help. Sadly v4l-request upstreaming seems to have stalled. I will try to contribute to it if it is not dead yet. I am especially fond of the v4l-request to vaapi bridge that could enable to share code with the intel vaapi support that is already in (sadly this bridge is the part that looks completely dead development-wise). It looks to me like v4l-request does the decode/encode job (hantro vpu and rockchip hw decoder. But I know little about rockchip RGA code. Might be this part cannot be supported by v4l-request only.

[HeyMeco]

linux 6.4-rcN

I hate to quote a meme but... That's the neat part, you don't!

There has not been a single commit to their kernel repo in forever - let alone a new branch to support anything newer than 5.10.x

In fact, here's my current one on a RK3588 (FriendlyElec NanoPi R6s)

# uname -a

Linux FriendlyWrt 5.10.110 #1 SMP Sat Dec 3 01:25:15 CST 2022 aarch64 GNU/Linux

This corresponds to https://github.com/rockchip-linux/kernel/tree/develop-5.10

Well Friendlyelec just updated their Rockchip branch to 6.1

https://github.com/friendlyarm/kernel-rockchip

[nyanmisaka]

This repo is rarely updated.

• https://github.com/rockchip-linux/kernel

Use these instead. This mirror is maintained by Radxa.

• https://gitlab.com/rk3588_linux/rk/kernel/-/commits/develop-6.1
• https://gitlab.com/rk3588_linux/rk/kernel/-/commits/develop-5.10

[nyanmisaka]

The current MPP module has two parts: the userspace libmpp.so in the repo and the mpp_service in kernel drivers/video/rockchip/mpp. This framework simplifies the kernel driver and keep most complicated and flexible part in userspace libmpp.so.

We use this framework in consider of our product requirement. We need to support multiple kernel version and have to update custorm's very old version firmware to fix issues. The kernel update is quite hard for already mass-produced firmware so we choose to update libmpp.so in userspace for custerms with different kernel version. This reduce our workload a lot.

About the kernel v4l2 driver it is quit hard to adapt to diffferent user requirements for example fast display on start playing. And also move all video codec protocol handling code to kernel is very hard to be accepted for kernel maintainers.

So base on these historical and realistic consideration we do not develop a common v4l2 like driver in mainline kernel. But for next generation MPP driver we are considering move the whole driver to kernel.

The above quote is from RK developer. They will not spend time working on the upstream v4l2-request. Because this interface cannot meet their actual needs.

To be honest, I don't have a good impression of v4l2-request, given that the RK3399 released 7 years ago has not received V4L2 encoding support yet. And actually the encoder API is not finalized. Not to mention the RGA part.

Then I got in touch with RK developer to see what they thought about native Libva/VA-API support. He said they would look into it.

[nyanmisaka]

Given that there will not be a better transcoding solution than MPP+RGA in the upstream in the short term, it makes sense to continue investing in MPP and RGA.

So I have a WIP branch with HW Decoding + HW Scaling + HW Subtitle-burn-in + HW HDR Tone-mapping enabled. This is an implementation based on MPP, RGA and Mali OpenCL blob.

TODO:

• Enable async RGA
• Polish MPP encoder and make it async
• Support AFBC to save DDR bandwidth

[wnpllrzodiac]

@prahal Check these drivers and patches: https://github.com/istoreos/istoreos/tree/istoreos-22.03/target/linux/rockchip/rk35xx/files/drivers/video/rockchip https://github.com/istoreos/istoreos/tree/istoreos-22.03/target/linux/rockchip/rk35xx/patches-5.10

Maybe someone can port to linux 6.x?

yeah,Iam interesting about it.only istoreos support mpp codec now. armbian openwrt didnt.

[prahal]

@nyanmisaka I agree that v4l-request development has mostly stalled (though I found no clue why, Collabora which was behind the development is doing great, maybe they had other higher priority targets). Conversely, MPP will never be included in any Linux distribution ... as it is only implemented in rockchip fork of linux which is years behind the mainline. They can follow the LTS but even then a distribution will ship with the current mainline when it is released, not a kernel that was released 6 years ago and has only had security patches.

So MPP is a rockchip BSP-only development of no help to anyone not running this fork. It is their call to focus on their stack for their fork. But this is not a Linux stack. It is a rockchip stack, plain forked. I never saw any intent from them to bring MPP upstream, or even make it an out-of-tree option against Linux. And it is not an achievable or even worth the effort task to constantly port their MPP moving target to mainline.

So for my use case, it is v4l-request or nothing. If you know of any project targeting Linux I would be pleased to give it a try.

Seems there are still living foks of the v4l-request Collabora works, but to me, they abandoned the v4l-request VAAPI bridge (that would have made integration into Jellyfin a not-so-hard task, except the jellyfin VAAPI implementation seems somewhat hardcoded for Intel, but I believe this can be fixed).

So I agree with you that v4l-request is stalled though with working manageable patch sets. But for people not running Rockchip OS I see no other option.

I see no point in implementing use space if no one has the kernel space required by this userspace. As far as I know, the most advanced ARM distribution with regards to patching the Linux kernel with various patchsets is LibreElec. If MPP is not even there ...

Also, the rockchip dev comment mostly states that it is too hard to upstream patches in v4l (for codec for one). So I suspect they lack the time to do so. But I doubt it is any easier to upstream patches anywhere in the Linux kernel. I guess they do not intend to upstream anything. In fact, it is way more difficult to get a new stack accepted in the Linux kernel (near impossible, especially in the media tree, see the Empia developer tv debacle. And this developer had already been involved in the media stack as a maintainer. Only he wanted to rip off the existing stack and start afresh) rather than getting anything else accepted.

Another point in favor of v4l-request is that it is not rockchip specific. A better option would be another framework that replaces both vaapi and v4l-request. I see v4l-request as a stepping stone. MPP could be a proprietary stack like QSV for Intel, AMA for AMD, or NVENC/NVDEC for Nvidia. But v4l-request is more of an attempt to build a vaapi that can cope with the split hw that is common on ARM boards. It would be even better development time-wise if it was shared with x64 hardware.

[nyanmisaka]

@prahal

Conversely, MPP will never be included in any Linux distribution ... as it is only implemented in rockchip fork of linux which is years behind the mainline.

For me, a kernel with a complete hardware feature set and stability is more important than pursuing a bleeding edge kernel all the time. I've been bitten more than once on the bleeding edge Intel i915 and AMDGPU drivers over the past few years. Also, compared with x86_64 hardware, the improvements in the mainline kernel of embedded ARM hardware used for desktop purposes are not worth mentioning. Unless your use case is OpenWRT or LibreElec. Of course, I agree to convert the driver in the downstream BSP kernel into a form acceptable to the upstream and submit it.

They can follow the LTS but even then a distribution will ship with the current mainline when it is released, not a kernel that was released 6 years ago and has only had security patches.

https://www.cnx-software.com/2023/11/02/rockchip-roadmap-reveals-rk3576-and-rk3506-iot-processors-linux-6-1-sdk/

But this is not a Linux stack. It is a rockchip stack, plain forked. I never saw any intent from them to bring MPP upstream, or even make it an out-of-tree option against Linux.

People in the Linux media only want V4L2. About 7 years ago, a former Rockchip employee discussed upstreaming MPP into a VA-API driver on the mailing list, and an Intel employee answered his doubts. But upstream developers have been promoting V4L2, which led Rockchip to convert MPP into a V4L2 driver and submit it as a draft, but V4L2 was just incompatible with MPP usage, so Rockchip stopped working on V4L2.

And it is not an achievable or even worth the effort task to constantly port their MPP moving target to mainline.

This is not unachievable, because someone has already done it, and I have tested it in the mainline 6.7-rc1 kernel. No one had done this before because the hardware specs were not attractive, but the RK3588 is apparently an exception. Also this is not unsustainable as the driver has been stable for many years, just not in the upstream repository. Rockchip has no intention of refactoring it, but only commits new hardware support and bug fixes.

Seems there are still living foks of the v4l-request Collabora works, but to me, they abandoned the v4l-request VAAPI bridge (that would have made integration into Jellyfin a not-so-hard task, except the jellyfin VAAPI implementation seems somewhat hardcoded for Intel, but I believe this can be fixed).

I personally refuse to waste time on any form of shim and wrapper of video driver, this is just a stop-gap. Rather than doing this, it would be better to rewrite it to provide native support, so that subsequent hardware can benefit from it. Rockchip has written a similar bridge before, but like the v4l2-request-vaapi bridge, there is no follow-up.

MPP could be a proprietary stack like QSV for Intel, AMA for AMD, or NVENC/NVDEC for Nvidia.

MPP is as proprietary as QSV but not closed source. Its kernel driver and user driver are open source, which is a very important reference code for rewriting the upstream video/DRM driver. Both AMF and NVENC are proprietary closed source products.

https://github.com/rockchip-linux/mpp/issues/405#issuecomment-1759212785

My idea is to convert the MPP & RGA kernel drivers into part of the upstream-acceptable DRM/KMS driver, and add their user-mode drivers to Mesa, sharing the same Libva front-end as the existing AMD RadeonSI driver.

At the same time, I also completed the checklist for integrating MPP&RGA into FFmpeg that I mentioned a few months ago. It's non-trivial but worth it. 1080p@480fps/4k@120fps encoding, subtitle burn-in, and HDR tone-mapping are all available without breaking a sweat.

[prahal]

I see a benefit in having an MPP stack for a little while as having developers consider that the media framework on ARM is of interest to users.

But telling MPP could get upstream is a fantasy. Simply because it is against all that upstream has told about requirements to be integrated. And the media framework is the most conservative I know in the kernel in this regard. And, adding duplicate stacks is against Linus' policy. The fact that the current framework does not support one's hardware fully was never a valid argument to add a duplicate framework. They want incremental changes or the new framework to support all the hardware the older one had support for. As far as I know, it happened for the USB stack that was replaced but it was early in the older stack integration.

QSV and I guess AMF and NVENV are either based on already upstreamed kernel interfaces that are based on v4l2 or require out-of-tree kernel drivers.

I was not saying that such a driver port cannot be done once. And even if MPP does not change, the kernel does. I am saying that it cannot be done for 10 years out of spreading the burden to 20 developers. And with such an amount of work mainline could be made to fully support what MPP does.

About v4l2. This is not a shim. This is a framework the drivers are built from. there are helpers, shared functions, and interfaces. v4l-request is an interface to the kernel. MPP is a full stack in the kernel. Mind I still do not know that much about MPP and have a lot to learn v4l2 side.

QSV is not a closed source either. AMF and NVENC are. QSV does implement a common layer for multiple platforms in userspace. But it uses v4l2 in the kernel. About AMF I do not know but I guess it does the same. NVENC knowing Nvidia's willingness to hide everything, should be an out-of-tree driver I suppose (if they did not add a GPL shim around a binary blob).

If you deem a new kernel of interest because it adds new features ... the fact thousands of bugfixes and refactoring go on. The idea is to improve reliability, speed, and adding support for new hardware. I don't want to be stuck because of hardware media support. The main idea is to share work. And port the drivers at the same time. The mainline drivers that is. What you call bleeding edge is bleeding stable. Regression happens but hardly. Features are very rarely added. The older the worse.

If MPP ends up getting enough developers to foster a sustainable team to bring it up to the latest kernel for 10 years (and I expect longer than 10 years of hardware support I use with Linux, even my devel laptop is 10 years old).

So the libmpp userspace lib to support multiple kernels only makes sense if they want to provide a dedicated userspace API to their kernel interface to the code that each application using MPP will have to implement (mostly ffmpeg and gstreamer though).

QSV provides a common API for Windows, Mac OSX, and Linux. Thus the requirement for a dedicated library. But even then QSV uses v4l2 kernel interface.

Rockchip devs discussed with upstream about the v4l2 API. But I never saw their reply to the other kernel developers in https://lkml.org/lkml/2019/3/7/517 . They were given clues and asked rationales and did not reply. To me the main issue is grammar. The rockchip developer sentences are pretty hard to understand:

> I would make something to describe a buffer with some addtional meta
> data.
> 
> But the current request API limit a buffer with associated with
> an request buffer, which prevent sharing some sequence data, but it 
> still can solve some problems.

It is known that the kernel framework needs work to support more discrete devices on the ARM boards. But ghosting won't help bring anything forward. And telling this is MPP or nothing will get them nowhere.

All the exchanges you shared with me gave a picture of kernel developers being willing to change the v4l2 API but requesting rationales and initial drivers. They have been ghosted.

To sum up there is no complete MPP stack now the kernel upstreaming later. There is no way that MPP upstreaming could get accepted in my opinion.

When they say in https://github.com/rockchip-linux/mpp/issues/405#issuecomment-1569478900 that they ship the code in libmpp and that most likely the MPP kernel code is just a shim to the hardware, to help with deploying the update to their customer firmwares with various kernel version, what they mean is they moved all the kernel code in userspace so that they do not even have to have their customer to have to update their kernels at all to get the fixes for their MPP logic. They do not understand that their BSP is built on the work of others. This is not you asking us for 100 hours of work to get our driver included and is too expensive. It is we took one thousand hundred hours of your work and do not want to pay a developer to spend 100 hours bringing our code into yours, contributing a portion of this time to help bring the kernel framework up to speed.

To sum up. Yes, v4l2 needs work. But at least it can be built upon. MPP is a dead end (except if you want to maintain a fork for 30 years ala Nvidia, but it is way more costly in the long run than bringing v4l2 up to par). Either v4l2 is built upon or rockchip support will never happen in mainline. I do not say that it is impossible that rockchip media HW ends up never being supported in mainline. But that would be a sad thing. Working on MPP in the long run only brings this closer to reality. In the short run, it looks cool but it is shooting itself in the foot.

There seems to be a misunderstanding. Why do you call v4l-request a shim? Or are you talking about eh v4l-request vaaapi shim that was since abandoned to leave only the ffmpeg v4l-request direct support (that sadly has been left without upsteaming effort for more than a year)?

Mind what I am afraid of is a new kernel device file interface specific to rockchip media HW. And not improving the kernel frameworks. Having code in userspace is fine. I am also not saying that using the existing v4l in-kernel framework is a must. Only that using or improving an existing kernel framework is. Or else at least provide an upgrade path to port other hardware drivers to it. It is all about the maintainability of the kernel code as a whole, not just making the life of a specific driver maintainer easier while increasing the work of all the other developers. To me, rockchip devs couldn't be bothered to contribute anything out of their own driver code. They do not even want to pay other developers to do the work required to get the functionality they need (Nicolas Dufresnes who asked them for more details is one such developer as far as I know, I do not even think he is a maintainer in the kernel team, but a consultant. He offered to help them freely and they ghosted him).

Edit: might turn out that the way you envision mpp in the DRM subsystem could get merged. I am not versed enough though I wonder why MPP requires a dedicated device file while QSV does not.

[HeyMeco]

@nyanmisaka

At the same time, I also completed the checklist for integrating MPP&RGA into FFmpeg that I mentioned a few months ago. It's non-trivial but worth it. 1080p@480fps/4k@120fps encoding, subtitle burn-in, and HDR tone-mapping are all available without breaking a sweat.

Would you mind sharing the commands/settings you used to get those results? I just build your ffmpeg branch and wanted to compare the results with the rk3568 soc I have here rn. I would love for you to maybe write a blog post about this as it would be interesting to many people

[nyanmisaka]

@HeyMeco

My results are specific to the RK3588 soc, which supports up to 8k@30fps encoding speed as per specs. You can check the datasheet to find the expected maximum performance on other socs.

RK3588: (8k@30 ≈ 4k@120 ≈ 1080p@480) RK3568: (1080p@100 ≈ 4k@25)

The latest branch is this one. It has been refactored many times and i don't want to drop the proto type branch.

# full hardware transcode to h264 1080p 6000k
/usr/lib/jellyfin-ffmpeg/ffmpeg -hwaccel rkmpp \
-hwaccel_output_format drm_prime -afbc rga -i /path/to/input.mp4 \
-vf scale_rkrga=format=nv12:afbc=1:w=-1:h=1080 -c:v h264_rkmpp \
-b:v 6M -maxrate 6M -g:v 120 -c:a copy -sn -y /path/to/output.mp4

# query decoders and decoder helps
/usr/lib/jellyfin-ffmpeg/ffmpeg -decoders | grep rkmpp
/usr/lib/jellyfin-ffmpeg/ffmpeg -h decoder=av1_rkmpp

# query encoders and encoder helps
/usr/lib/jellyfin-ffmpeg/ffmpeg -encoders | grep rkmpp
/usr/lib/jellyfin-ffmpeg/ffmpeg -h encoder=h264_rkmpp

# query filters and filter helps
/usr/lib/jellyfin-ffmpeg/ffmpeg -filters | grep rkrga
/usr/lib/jellyfin-ffmpeg/ffmpeg -h filter=vpp_rkrga

I can write a more detailed docs later but now i have to finsh my changes in jellyfin server.

[HeyMeco]

@nyanmisaka wow thank you for that elaborate resonse. I got it working with RK3568 without any more modifications this is exciting

[RadxaYuntian]

@nyanmisaka #160 seems to go nowhere. Do you plan to create a PR for jellyfin-ffmpeg?

[CodeChenL]

@nyanmisaka Looking forward to your docs!

[IngwiePhoenix]

The definition of a 10x developer and a giga chad. This is amazing and I am not even memeing.

Thank you so much for your hard work @nyanmisaka ! Looking forward to the merge, this single handedly makes a huge amount of RK35588 and other RockChip based boards viable as Jellyfin boxes and I am so excited.

[RicardoCst]

Is the pull request that will close the issue only for RK3588, feels kind of futureless? This will cause the same issue as with the ODROID XU4 Samsung Exynos 5422 chip. But otherwise this is great that theres finally a recent ARM board that has hardware support in jellyfin.

[HeyMeco]

Is the pull request that will close the issue only for RK3588, feels kind of futureless?

As far as this goes with my tests it works with RK3568 too so and should with all future Rockchips

[nyanmisaka]

Is the pull request that will close the issue only for RK3588, feels kind of futureless? This will cause the same issue as with the ODROID XU4 Samsung Exynos 5422 chip. But otherwise this is great that theres finally a recent ARM board that has hardware support in jellyfin.

Rockchip's implementation is actually platform agnostic. I could only verify on RK3588 boards so that's the only one I mentioned. Theoretically it works on all RK35xx and older RK33xx.

[zhu327]

Is there a Jellyfin Docker image that supports MPP? I'd like to try it on RK3399.

[HeyMeco]

Is there a Jellyfin Docker image that supports MPP? I'd like to try it on RK3399.

You should be able to link

• /usr/lib/jellyfin-ffmpeg/ffmpeg to /bin/ffmpeg

and do the same for ffprobe and also link

• /dev/rga
• /dev/dma_heap to the same path.

I‘ve attached a precompiled deb for @nyanmisaka‘s fork. jellyfin-ffmpeg6_6.0.1-1-bullseye_arm64.zip

I‘m not sure about the exact settings in the Jellyfin UI or if i’m missing something. Maybe the card0 device. But just to test it in the command line (not docker) this deb worked for me.

if transcoding fails try deleting /dev/ion on the host (no restart needed) source @jjm2473

[nyanmisaka]

I‘ve attached a precompiled deb for @nyanmisaka‘s fork. jellyfin-ffmpeg6_6.0.1-1-bullseye_arm64.zip

Prebuilt deb can be downloaded from

[RicardoCst]

You are awesome!

This will make jellyfin unique in the way of support for arm64 / armhf! Meanwhile Plex and Emby are whining about how much time and energy and cost it takes to implement more hardware support...

[nyanmisaka]

Here's a docker image for testing. Feedback is welcome.

docker run -d \
 --name jellyfin \
 --privileged \
 --net=host \
 --restart=unless-stopped \
 --volume /path/to/config:/config \
 --volume /path/to/cache:/cache \
 --volume /path/to/media:/media \
 `for dev in dri dma_heap mali0 rga mpp_service \
    iep mpp-service vpu_service vpu-service \
    hevc_service hevc-service rkvdec rkvenc vepu h265e ; do \
   [ -e "/dev/$dev" ] && echo " --device /dev/$dev"; \
  done` \
 nyanmisaka/jellyfin:latest-rockchip

[HeyMeco]

Here's a docker image for testing. Feedback is welcome.

@nyanmisaka Can you allow "issues" on your fork page? So we don't spam this thread.

I tested some content formats and most seem to work fine as in H264 normal and interlaced @50FPS (still with RK3568 not RK3588) but having HEVC HDR content as source and HW acceleration for it turned on leads to weird line artifacts.

Edit: This seems to be an encoding issue for settings above >1080P@15mbps H264

Edit 2: using HLS and HEVC it works fine but the 1080p encoder only is capable of 15fps@40mbps so and 35-40fps@20mbps The RK3568 is fine for 1080P then, interested to see the RK3588 results from you guys and hopefully soon from me too.