Support live transcoding

[nsrosenqvist]

Are there any plans of implementing support for live transcoding of unsupported file formats/codecs?

[Instagit]

We've had a few discussions about this, most recently https://github.com/nextcloud/files_videoplayer/issues/40, but every solution has significant drawbacks.

I think when the JS solutions are stable and fast enough for productive use, video players will implement them or provide plugins, so there's not much on our side to be done, unless someone wants to help directly with the development of the transcoding libraries.

[nsrosenqvist]

Ok, and there's no interest in server side transcoding? I guess that wouldn't work for enterprise due to the amount of processing power required

[Instagit]

Yes, that's the problem. We don't collect statistics, but many Nextcloud instances are most likely running on low powered devices. First of all, many won't have the computing power to transcode HD videos at 25 FPS and even if they did, it would be unreasonable to use up so many server resources without the user knowing. And then there are security aspects of starting system background processes via PHP.

Nextcloud is extendable, so if someone really wants this feature, they could implement it and upload it to the app store. That way, users who don't mind the drawbacks can use it.

I'll close this, but if there are any more questions feel free to ask.

[tilllt]

I think the argument is valid if you think about Nextcloud instances. My guess would be that the majority of nextcloud installations is used by one or very few persons. And the majority of processors feature hardware accelerated transcoding, even the RPi Arm processors, not to speak of Intel Quicksync and Nvidia NVEnc. ffmpeg supports these hardware accelerations so that should really not be the issue.

[Instagit]

@tilllt

>the majority of processors feature hardware accelerated transcoding, even the RPi Arm processors, not to speak of Intel Quicksync and Nvidia NVEnc

We don't collect user statistics, so we really don't know what hardware Nextcloud runs on. You can run it on a Raspberry Pi.

"Should not be an issue" is not a good approach to developing enterprise quality software. Try live encoding a full HD or 4k video, maybe even one with 60 fps. It takes a lot of resources, even with good hardware, and doing this by default would be completely unreasonable.

My Ryzen can't even hit 60 fps for 4k videos and how many servers do you know which have fast GPUs? You also need to consider that several users can watch videos at the same time. And when we ignore the performance, there are still the security implications. FFmpeg is not known for having a great security record (not blaming them, just saying - multimedia codecs are difficult to work with in that regard).

It's easy to ask for a feature that suits your use case, but I hope you can see that there's a lot more to consider.

[tilllt]

I dont know if you skipped this part of my answer: the ARM cpu in RPi's DOES support of hardware transcoding, as do most intel CPU's, NVidia GPU's etc.pp.

And no, maybe you didnt understand the point: thoses systems have a dedicated part of the hardware for video transcoding. So if running on a supported system the transcoding almost takes NO RESOURCES at all, thats the whole point.

The GPU does not need to be fast. Id suggest you read the article on QuickSync (Intel) or NVEnc (Nvidia)

https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video https://en.wikipedia.org/wiki/Nvidia_NVENC https://trac.ffmpeg.org/wiki/HWAccelIntro

[Instagit]

@tilllt

The Raspberry can only handle 1080p and that is for ONE user (still impressive, but not good enough to rely on it for a media server). And as I already pointed out, even if every CPU supported it, there'd still be problems that make this very difficult to implement properly.

We would need an implementation that

• offers seamless integration for both systems with and without hardware encoding
• offers a consistent user experience
• can dynamically switch between hardware transcoding and normal streaming when the hardware transcoder is maxed out... which would directly contradict the previous point, but is still better than stuttering or crashing
• doesn't cause any kind of hardware problems like overheating
• doesn't interfere with other services running on the server
• has no security implications, i.e. syscalls to FFmpeg (we would probably have to sandbox the transcoder)
• doesn't make it more difficult to set up Nextcloud

From my point of view, it's currently impossible to do with these requirements, but feel free to point out technical solutions. I'd really like to have this feature in Nextcloud if it didn't interfere with anything else.

Have you tried Plex? It's a dedicated home theater software that has transcoding built in, although with restrictions. Select the right tool for the job etc. ;)

[tilllt]

I think there is a lot of potential for all kinds of transcoding approaches, if you look at:

• hw accelerated javascript: http://gpu.rocks/
• handbrake in js: https://github.com/75lb/handbrake-js
• ffmpeg in javascript: https://bgrins.github.io/videoconverter.js/
• php video transcoding: https://github.com/riyaskp/PHP-Video-Transcoder

i think as an intermediary step it would be good if nextcloud could hand off the handling of certain file types - unless thats already possible.

[Instagit]

Instead of providing technical feedback about my concerns, you seem to link to random projects that have to do with transcoding. That's not helpful. Not only do they not provide solutions to the problems I mentioned, but some of these are virtually unknown and untested projects. You can't seriously expect them to be used in Nextcloud.

I told you very precisely why I think transcoding is currently not a possibility. I'm not getting paid for this and I'm spending my private time here, so unless you can give in-depth technical approaches for the requirements I stated, I have better things to do and consider this discussion to be over.

[tilllt]

Well, you only said its not possible for multiple users to have transcoding at the same time on a low-performance platform, which i consider self explanatory. On some low-performance platform transcoding will not be viable at all, also self explanatory. As far as i can see, Nextcloud uses video.js, so the player is not even coded by the nextcloud team itself. So, looking at what others projects are doing in terms of realtime transcoding, might be more fruitful then discussing if someone here can implement hardware transcoding - which sorry to doubt your skills - i think you cant. neither can i. so i dont understand why you are argumenting as if you COULD. Why not look at other projects that actually are closer to doing what we discuss here then discuss if video.js could support video transcoding - which it clearly can not. Because it does not mean there can not be a way to realtime transcode video, given a minimal set of requirements - on the server.

[tilllt]

So to make it short: would it make sense if a self hosted cloud application would support realtime video transcoding, so you dont need an extra media server? I guess we all agree that it does.

Is it possible with the current software used for serving video in nextcloud? Probably not.

But that does not mean it doesnt make sense to think out of the box and consider options on HOW it could be possible.

[heathfx]

it's 2021 now and plex does on-the-fly transcoding masterfully when given access to an nvidia GPU, so I know this can be done. Because i use my server first and foremost as a 10gbps samba server for video editing, it already has the "beans" to handle a heavy workload. Being able to share footage with someone else would be made much easier if they could see previews immediately vs. waiting for background encoding tasks.

I already allow a proxmox linux container running plex to access my GPU and it can transcode BD rips in realtime with no hit to the CPU. I also give access to the GPU and a coral TPU from a container running frigate, it runs a tensorflow model over surveillance camera footage to detect humans appearing on my cameras...again with barely hitting the CPU.

People with a use-case for fast on-the-fly transcoding will go out of their way to meet specialized hardware needs.

[pktiuk]

Real-time transcoding to different quality would be really useful. (I guess it would tackle https://github.com/nextcloud/files_videoplayer/issues/187 )
Are there any plugins/workarounds allowing this functionality? I store high quality videos on my NC, but I don't have a very good banwidth.

EDIT
I found memories, which seems to have video transcoding https://github.com/pulsejet/memories/wiki/Configuration#video-transcoding
And it works very well

[pktiuk]

I think this issue should be reopened and reconsidered (some things changed since 2018 :) )