Metal Translation

[patriciogonzalezvivo]

@vade and @apsalin started working on this PR for a translation for Metal https://github.com/patriciogonzalezvivo/lygia/pull/140#issuecomment-2227328478

I'm opening this Issue so we can merge that initial effort and use this place to have conversations about the strategy and specs for Metal.

My suggestion for handling samplers is:

• declare a default in sampler.mls following sampler.glsl pattern * define overloaded function declarations which allow users to override the border, mip map and wrapping calls while providing sane defaults. Look into: https://github.com/patriciogonzalezvivo/lygia/blob/main/sample/repeat.glsl https://github.com/patriciogonzalezvivo/lygia/blob/main/sample/zero.glsl https://github.com/patriciogonzalezvivo/lygia/blob/main/sample/clamp2edge.glsl

Which allows in GLSL that samplers can be "over-writen" per function using #defines. Ex. : https://github.com/patriciogonzalezvivo/lygia/blob/main/filter/bilateral.glsl#L18

[vade]

Hey

I was able to gt Lygia linked and working in a live code Metal App im working on, which now makes it SO MUCH easier for me to validate porting and test things. Most of this will be driven completely by my own self interest and be in no particular order. YMMV etc etc.

Progress on the Metal Port (as well as issues) is tracked in the README_METAL.md in the PR (which will eventually land in main)

[patriciogonzalezvivo]

Awesome @vade! Self interest it's 100% the name of the project. I'm really happy that you have something that is useful and work. Without that is hard to muster any energy/interest. Yay for proving that just works and is possible. Fully excited!

[apsalin]

Hey @vade did you end up setting up lydia manually, without using SPM?

[vade]

Yea, basically in my setup right now, im using include paths in my metal shaders I load / compile at runtime (thanks to Satin from @rezaali )

I think doing SPM is going to be next to impossible without a breaking set of changes because SPM sort of enforces some directory structure stuff as far as I can tell?

[vade]

Did some work in #197 which gets some very basic sampling set up.

ill be sort of ad hoc going through shit slowly as I need it.

If anyone else wants to do a more systematic port it's pretty straightforward. Check the README_METAL.md for my porting methodology (or lack there of ha)

[vade]

hey @patriciogonzalezvivo - is there any reason all the languages are in the same folders? It might be helpful to understand what is/ isnt ported by being able to compare top level file trees like

"glsl/filters/*.glsl"

vs "cuda/filters/.cu" "hlsl/filters/.hlsl" "metal/filters/*.msl"

etc

[patriciogonzalezvivo]

oh... interesting. didn't thought it that way, I founded easier to see files side by side. If you want to see all the files from one single language you could use the server:

https://lygia.xyz/glsl.json https://lygia.xyz/hlsl.json https://lygia.xyz/msl.json https://lygia.xyz/cuh.json https://lygia.xyz/wgsl.json

fun fact about the server it can return files with satisfy dependencies. Ex: https://lygia.xyz/math/atan2.glsl

you can get language specific meta data by adding ?lang=<extension>. Ex: https://lygia.xyz/math/mix.json?lang=hlsl

or the metadata https://lygia.xyz/math/atan2.json

Also related. I have mean to add some sort of script that remove all languages and keep only one. So people interested on bundling lygia can do that. And have release versions that way. Just GLSL, just HLSL, metal, WGSL, CUDA, etc.

Is that the question?

[vade]

Oh no, was more of an organizing / adhd principle. I just find all the files in a single directory a bit cumbersome for my brain is all haha.

all good, it would be a large change, I just like being able to a / b two folders side by side to see whats what.

[patriciogonzalezvivo]

oh got it! Yeah it gets hard to follow. Sorry.

ONE-MORE-NOTE on the server: the long term idea for this kind of API https://lygia.xyz/color/blend/screen.json?lang=msl is to be able to populate the catalog of a node graph editor ; )

the server is doing some very poor parsing. But if you are interested we could put more love in this

[shadielhajj]

Interesting points @patriciogonzalezvivo and @vade. As a Lygia DEV, I can see arguments for both approaches (single folder vs. per-language folder), however as a Lygia USER, I really want to have only the language I'm interested in. For example, it's very cumbersome to have .cuh and .glsl files in a Unity project. Sadly using the server is not an option in many cases (installation work, game engines, etc...). Nothing urgent in any of this, but an interesting point to discuss in the future, IMO. Maybe the script you mentioned @patriciogonzalezvivo would be a good option.

[patriciogonzalezvivo]

Yes, I think that script + per release with per language zip files + independent CI testing.

We have lot to entertain our selfs : ).

Would be nice to have per language leads. @shadielhajj have been championing HLSL, @vade it's the soul contributor to METAL and @sdedovic started working on bringing WGSL to parity (WGSL is really different, it's challenging lot of the design decisions).

Let me know if you are interested on being the official lead for each language.

[shadielhajj]

Yes, happy to take the lead for HLSL @patriciogonzalezvivo :-D

Speaking of which. One thing that has been bugging me for a while is the sync between languages. I've been updating HLSL in a very ad-hoc manner and have no idea of what needs to be done to bring it to parity with GLSL. Some automatic tools would be ideal, not sure if that exists. Obviously, the tool would need to keep track of every change to the reference version (GLSL)

Again, nothing urgent. But I feel sync between languages is going to become an issue at some point.

[vade]

Did a pretty large PR of math / draw / SDF metal shaders. I have not vetted most, but spot checks seem to work? 🤷‍♀️

[vade]

Need to proxy some GLSL 'compat' functions in some sort of include / compat header.

lessThanEqual, greaterThan, greaterThanEqual, equal, notEqual, any, all, not

[vade]

mod as well.

[vade]

Need to handle inout for metal. can make thread, thread group or device scoped memory that requires a reference.

[vade]

Need to figure out float4 fragCoord [[position]] for gl_fragcoord compatibility

[rezaali]

"So [[position]] coming out of the vertex shader contains the float4 position of the vertex, but going into the fragment shader, the xy of the [[position]] contains the screen coordinates of the current fragment."

Ref: https://forums.kodeco.com/t/screenspace-uv-coordinates-in-fragment-shader/112236

[vade]

Yea @rezaali but I meant more of a clean way to shim in all of the assumed calls to gl_fragCoord.xy in fragment programs in a nice clean way so that metal shaders can have the same functionality. I think the [[position]] style variables have to be declared in the main fragment shader code function declaration?