search

pabloromeo / clusterplex

ClusterPlex is an extended version of Plex, which supports distributed Workers across a cluster to handle transcoding requests.
MIT License
467 stars 36 forks source link

Clusterplex Worker Protocol Documentation -- Allow 3rd party workers to integrate with Clusterplex #153

Closed donicrosby closed 2 years ago

donicrosby commented 2 years ago

Is your feature request related to a problem? Please describe. I'm currently trying to setup a Clusterplex instance in Kubernetes using NVIDIA Jetson Nanos. NVIDIA in their infinite wisdom does not support normal hardware acceleration for video and audio codecs (i.e. CUDA) on the Tegra GPUs as they have their own dedicated hardware for video encoding and decoding. This also requires a special version of FFMPEG that cannot be bootstrapped in the normal Linux Server Plex Container as again in NVIDIA's infinite wisdom uses Ubuntu 18.04 as the most up to date supported version for the Jetson and when trying to compile the Jetson Supported FFMPEG in the container, the built binary fails to start due to incompatibles between Ubuntu LTS releases.

My solution is writing a compatible worker that is domain specific for the Jetson (https://github.com/donicrosby/clusterplex-jetson) so that other people can use your awesome software to build really powerful plex servers with these powerful boards. Because the Jetson supports HVEC and x264 on hardware with specific codecs the worker would replace the -codec:v of the transcoder with the hardware accelerated ones so that none of the transcoding is done via the CPU.

Describe the solution you'd like Just a simple README on what JSON objects need to be passed to the Orchestrator over the websocket and when possibly some diagrams (I know that's asking for a lot...) so that people who have specific hardware like the Jetson can use Clusterplex workers on them.

Describe alternatives you've considered Add support in the worker to pass either a YAML or JSON config file with a "swap-out" mapping of video and audio codecs, this would mean the worker would look for the -codec:v and -codec:a options in the command and replace them with the possibly hardware accelerated ones. This would allow for 3rd party implementations of FFMPEG to be pointed in containers after installing your worker.js and it should "just work"

Additional context None

pabloromeo commented 2 years ago

No need to close it. It sounds like an interesting idea, I just haven't had time yet to look into it further :(

donicrosby commented 2 years ago

So I managed to get a custom worker running, however Plex's version of FFMPEG is incompatible with normal FFMPEG. Even removing all of the unrecognized options the stream would not work unfortunately. So I closed the issue as there's no way to implement a 3rd party version of Plex FFMPEG as it's closed source :/

pabloromeo commented 2 years ago

Oh, really? That sucks. Could it have been due to the "progress calls" Plex's FFMPEG version does while transcoding? I know it makes periodic calls to the progressUrl that is passed as parameters. I remember seeing, some time ago, one version of Plex's ffmpeg build, and you could see the code where they make that call. Not sure if it's worth looking into. They technically have to make it available if you request it. They are bound by FFMPEG's open-source license to do so.

musicderp commented 3 months ago

So I managed to get a custom worker running, however Plex's version of FFMPEG is incompatible with normal FFMPEG. Even removing all of the unrecognized options the stream would not work unfortunately. So I closed the issue as there's no way to implement a 3rd party version of Plex FFMPEG as it's closed source :/

Sorry to necro this but plex's ffmpeg legally cannot be closed source and they provide (very hidden) downloads to their ffmpeg source https://downloads.plex.tv/ffmpeg-source/plex-media-server-ffmpeg-gpl-63e2f7282aa.tar.gz