leverage Vulkan as default on MacOS and iOS now that MoltenVK is open-sourced

[classicjazz]

Knowing all of @haasn hard work on Vulkan as well as the current development on cocoa-cb, I wanted to call out that the Khronos Group has annouced that MoltenVK is now open sourced (Apache license).

https://github.com/KhronosGroup/MoltenVK

MoltenVK is the library which translates calls within the Vulkan graphics and compute API, to underlying calls to Apple's Metal API on both iOS and macOS.

Khronos cites how Valve achieved 50% faster performance for one of its games using MoltenVK. I don't know if that is directly applicable to mpv.

https://www.khronos.org/news/press/vulkan-applications-enabled-on-apple-platforms

But, I thought it might permit, not only more efficient MPV performance, but also use of @bjin 's RAVU Vulkan prescalers on MacOS and iOS, including ravu-3x, which currently generates a blue screen using OpenGL on Mac.

https://github.com/bjin/mpv-prescalers/tree/master/vulkan

[wiiaboo]

What's the issue then? If you could rework the title and message to look more like a feature request than a news article, it'd be great.

[ghost]

Apache is still incompatible with (L)GPL.

[classicjazz]

Clearly, you can't contribute LGPL licensed code to an Apache licensed project.

But why can't an LGPL licensed program leverage (properly build and then invoke) an Apache licensed subsystem (where that code is separately maintained on Github, eliminating any issues about the applicable license for code contributions)?

https://github.com/KhronosGroup/MoltenVK#using

[Hrxn]

If something like this should really fail because of license crap... I'm jumping from the next bridge 'round here..

[Argon-]

Apache is still incompatible with (L)GPL.

Couldn't it be done similar to crossc/SPIRV-Cross for d3d11?

[haasn]

Whether or not Apache is compatible with LGPL is of no concern to us, because of the way vulkan is structured. We don't link directly to MoltenVK, the same way we don't link directly to RADV, ANV or Nvidia code. We're only using the vulkan API here, and the VK_MVK_macos_surface extension is part of the same vulkan API specification as the rest of the code we're using. Whether or not it happens to be MoltenVK underneath, or some other vulkan implementation, is an implementation detail of the platform.

In terms of implementing support, it would be relatively straightforward: You add a new --gpu-context type that uses VK_MVK_macos_surface in the same way that the other code uses e.g. VK_KHR_xlib_surface. That just requires the following object to be specified:

pView is a reference to a NSView object which will display this surface. This NSView must be backed by a CALayer instance of type CAMetalLayer.

If anything, the major issue is that, last I checked, MoltenVK is not a conformant vulkan implementation yet. mpv will simply not work with it until they implement the missing features (events, linear sampling, compute shaders, etc.).

[haasn]

Hmm, according to this article, event support is currently not part of the vulkan portable subset.

So it may be possible we have to make our usage of VkEvents optional (easy). It may impact performance on OS X though. Then again, I guess OS X users are used to being second-class citizens everywhere.

[ghost]

I'm by no means a license expert, but it does matter even if we dynamically load Vulkan. Besides, we still need to link to the khronos loader, which is apparently also apache licensed. All this doesn't matter on Linux because it can be argued to be a system library there.

[maru-sama]

Would this change actually result in much better performance or have other benefits? Up to now the osx output worked without any additional libraries since only system frameworks were being used. For Molten you would have to actually build the framework yourself make it available in your system and then link to it. I am not sure how it looks license wise if you are providing a pre-compiled bundle for download since you would also need to add the framework/library for it to work correctly. So apart from the actual source changes you will have a more "complicated" build chain as well.

[haasn]

Would this change actually result in much better performance or have other benefits?

The most interesting outcome would probably be feature parity. OS X's OpenGL is stuck on a very ancient version, and even that is hilariously broken beyond repair - not that Apple has bothered trying to do so. So currently most features just flat out don't work on OS X, especially stuff like the high quality HDR tone-mapping (which may be of interest to the color-oriented OS X users out there).

But that said, a performance boost is definitely possible.

[Akemi]

don't misunderstand me i am not going to block a feature like this but i think it's just plainly a bad idea. atm it still doesn't support everything we need and who knows what we will need in the future and if this would ever be added to MoltenVK. also correct me if i am wrong, this seems similar to angle, and we try to replace angle with a proper dx backend utilising crossc/SPIRV-Cross on windows, or maybe we could say it is already replaced(?) considering that it works well and has all the features. i bet a vulkan backend utilising MoltenVK would also need quite a bit of 'work arounds' because it most likely will not behave completely similar or is otherwise constricted by Metal.

to me it seems adding a Metal backend to vo=gpu utilising crossc would be wiser. it won't be more work, but probably less, it will most likely perform better but not worse, it already has the features we need to have a feature parity with the other backends, we have more control over what is exactly done, and considering Apple we will get into less fuckery looking at what they did with opengl and how they purposely make it perform horribly (i am implying that they might do the same with MoltenVK if they don't like it). the last part is just me assuming things and i can only back this up with the degradation of our opengl performance over the various macOS updates.

[ghost]

These don't exclude each others. Of course an emulation layer will always add overhead, bugs, or a reduced feature set (see ANGLE vs. ra_d3d11). So if someone wants to add Metal backend that'd probably be fine. On the other hand, ra_d3d11 tends to have problems due to the flaky and slow shader translation process.

In any case, you're probably the only one who could do that Metal work. It's either that, or just using MoltenVK (integrating this would also have to be done by you unfortunately).

[Akemi]

yeah, no exclusion. i was just implying that if someone wants to spend their time on this and is undecided whether to do one or the other, the time is probably best spent on a Metal backend.

MoltenVK also uses SPIRV-Cross(?), so it will probably perform similar flaky, if i am not wrong.

[haasn]

to me it seems adding a Metal backend to vo=gpu utilising crossc would be wiser. it won't be more work, but probably less,

Well you're welcome to give it a try, but if metal is anything like vulkan then I think you may be severely underestimating the amount of work that goes into a new RA backend versus the probably small handful of work-arounds we'd need for MoltenVK. (Such as disabling the usage of VkEvents, which is just one line change..). Each RA has around 3000 lines of minimum code needed to support one platform.

And like @wm4 said, trying to cross-compile GLSL to MSL is most likely going to be just as flaky as what MoltenVK is doing. (I mean, technically, that's what MoltenVK is doing). Cross-compiling GLSL to HLSL is already a nightmare, and those two languages are much similar than MSL.

[Hrxn]

I'm not really sure about the exact issue, but the slow shader translation process - with regard to d3d11 for example - is not some showstopping bug. I mean, you can't expect the same performance as with native GLSL obviously, but I think calling it "flaky" is a bit of an exaggeration.

[haasn]

I'm not really sure about the exact issue, but the slow shader translation process - with regard to d3d11 for example - is not some showstopping bug.

I'm pretty sure like 95% of that time (or more) is spent inside Microsoft's HLSL compiler. So the bad d3d11 shader compile performance is irrelevant to MSL unless Apple suck as hard as Microsoft when it comes to writing a compiler. (But then again, they just modified their $CXX)

[ghost]

Yep, they're essentially using clang. I have no idea how fast their compiler is, but probably not as terrible as Microsoft's.

I hope the D3D11 issue gets fixed by them adding native D3D bytecode output instead of HLSL, but we'll see.

[haasn]

I hope the D3D11 issue gets fixed by them adding native D3D bytecode output instead of HLSL, but we'll see.

In theory it might also be possible for AMD/Nvidia to get SPIR-V ingest into their windows drivers somehow. I mean the software and hardware both clearly support it.

[CounterPillow]

and considering Apple we will get into less fuckery looking at what they did with opengl and how they purposely make it perform horribly (i am implying that they might do the same with MoltenVK if they don't like it)

Hanlon's razor applies here.

Also, if Apple went out of their way to slow down MoltenVK by specifically checking for it in their code, I'm fairly sure this would open them up to an antitrust lawsuit or similar in other jurisdictions. Since Mr. Moneybags Valve is depending on MoltenVK for some of their income now, that would be a terribly unwise decision for Apple to make.

[Akemi]

Well you're welcome to give it a try, but if metal is anything like vulkan then I think you may be severely underestimating the amount of work that goes into a new RA backend versus the probably small handful of work-arounds we'd need for MoltenVK. (Such as disabling the usage of VkEvents, which is just one line change..). Each RA has around 3000 lines of minimum code needed to support one platform.

or maybe i am overestimating the work that has to be done to support MoltenVK. i also thought more about the long run. i didn't mean to belittle the work of an RA backend, if it sounded like it.

@CounterPillow it wouldn't be the first time for Apple going out of their way to slow something down.

[classicjazz]

I am not sure how it looks license wise if you are providing a pre-compiled bundle for download since you would also need to add the framework/library for it to work correctly. So apart from the actual source changes you will have a more "complicated" build chain as well.

I use HomeBrew on the Mac to ensure that both mpv and ffmpeg use the latest version from Github brew install mpv --HEAD mpv

My assumption is that you could name MoltenVK as a dependency. Then keep everything up to date via HomeBrew. Which would solve both the build chain issues and any licensing concerns.

Is there any sense of the level of effort for MPV to invoke MoltenVK so that we could test both performance and any issues? (I am particularly interested in higher performance 4K upscaling. Though feature parity would be a big win too...)

[DaveBinM]

With Apple now deprecating OpenGL in macOS 10.14, will mpv need metal or MoltenVK soon?

[Swyter]

It has been 'deprecated' for a long time, now it's official. I don't think the current GL drivers are going anywhere, even Photoshop uses OpenGL internally for pixel processing. They'd have to be mad to remove it.

[Akemi]

it's just deprecated and unlikely to vanish any time soon. tbh nothing really changed, since Apple ignored opengl for years already.

though yes, one day we will need something new. whether it is Metal, MoltenVK or something entirely different.

[DaveBinM]

I knew it hadn't changed in a while, but with it being officially deprecated now, I was curious what it would mean for mpv on macOS. 🙂

[CounterPillow]

It means you'll never get compute shaders because Tim Cook hates you, and you should probably stop giving them money.

[Argon-]

@DaveBinM no matter how much they deprecate, the fact that MoltenVK does not implement all of Vulkan 1.0 makes it useless to mpv. mpv uses Vulkan features MoltenVK doesn't offer.

[DaveBinM]

@argon- I still have a lot to learn 😅

[ldwardx]

A few months have passed, is someone implementing the metal backend? I am a developer of the Apple platform and a big fan of mpv. I plan to implement the Metal backend as a topic for me. Although I am not very grasp at the source code of mpv at present, I am willing to put most of my energy into this work. Any suggestions for this? Thanks.

[Akemi]

on a side note, issue #5088 is more applicable to that.

i started on a metal vo=gpu backend some months ago, though my master thesis and job hunting kinda got in the way. i've got the basic structure down, some pixel formats and basic integration with swift. i was planning to use swift for most of it if possible. shaderc for GLSL -> SPIR-V and crossc/SPIRV-Cross for SPIR-V -> MSL shader conversion similar to the d3d11 backend. if you want/need some more info feel free to poke me on irc, der_richter on #mpv-devel.

[ldwardx]

Thank you @Akemi . I hope to join this work if you don't mind, and I also like to use swift. I will contact you at IRC and look forward to working with you.

[Akemi]

i implemented the first POC #7482. use with --vo=gpu --gpu-api=vulkan.