jellyfin / jellyfin-ffmpeg

FFmpeg for Jellyfin
https://jellyfin.org
Other
436 stars 118 forks source link

[6.0] Add CUDA transpose/flip filters #346

Closed nyanmisaka closed 4 months ago

nyanmisaka commented 4 months ago

Changes

Shadowghost commented 4 months ago

How are these kind of filters not part of upstream FFmpeg?

gnattu commented 4 months ago

When will these filters be used? If we want the same functionality for macOS, we need to backport vf_transpose_vt as well. This filter performs both transpose and flip operations using VTPixelRotationSession, but it is only available on macOS 13+. This is newer than our targeted supported platform, macOS 12. The official projected end-of-life date for macOS 12 would be around October-November this year when the next major version of macOS is out, and we still have a few months to support it.

gnattu commented 4 months ago

It seems like we need to find some fallback path if our target is macOS 12:

gnattu@IntelMontereyVM Downloads % ./ffmpeg -version
dyld[555]: Symbol not found: (_VTPixelRotationSessionCreate)
  Referenced from: '/Users/gnattu/Downloads/ffmpeg'
  Expected in: '/System/Library/Frameworks/VideoToolbox.framework/Versions/A/VideoToolbox'
zsh: abort      ./ffmpeg -version

The binary uses VTPixelRotationSessionCreate cannot even be loaded on macOS 12, so we have to either make a build dedicated for macOS 12, or we use the fallback path for all platforms for now.

What functions are we needed? Only the flip or do we need the full transpose?

nyanmisaka commented 4 months ago

We are in no rush to fully enable it in 10.9 right away. You can optionally add this filter for higher version systems. I expect we will use the filer for 90,-90,180,-180 degrees. Therefore we need clock, cclock, reversal(flip) and even more.

nyanmisaka commented 4 months ago

How are these kind of filters not part of upstream FFmpeg?

There are no plans upstream to port commonly used software filters to hardware filters. They need to wait for contributors or bounty it as a GSoC project.

gnattu commented 4 months ago

I ported the transpose filter to CoreImage, and the code is published in this branch: transpose_vt_fallback.

This ported filter has the same name and supports all options of the upstream transpose_vt filter. Therefore, we do not have to change our server's code when we move to the upstream implementation and drop macOS 12 support later this year.