Metal Support - Offer to write the Heaps Driver

[onehundredfeet]

After building a variety of incomplete drivers for Heaps, a Vulcan one and one for 'the forge', I think that I'd like to volunteer to write the metal driver directly instead of going through another layer.

I'd like to make this a separate .hdll but still use the SDL. I've gone through this before and it requires more a bunch of #if #end blocks in the main heaps code.

Are you open to integrating metal for Hashlink?

Ryan.

[rcstuber]

Firstly I find it awesome that someone would offer to do such a big work! A few thoughts from my side though:

• Does anything speak against using MoltenVK?
• HL/JIT is still running x86 only, so we are tied to Rosetta on Apple Silicon which might not be around forever

So especially the latter point is a major pain point. Then we'd have Metal and still won't be able to develop (and debug) on modern Macs.

Cheers

Disclaimer: I'm not working for Shiro and have no say in this ;-)

[onehundredfeet]

Hey, thanks for the thoughts!

• Originally, I was going to use MoltenVK, but I've been warned against it by some professional engine writers that have used it before. Also, there's a lot of benefits to directly using Metal, without having a second layer to try to figure out why something isn't working. Metal has subtly different paradigms from Vulcan which can make the second layer more complicated than It needs to be. Using MoltenVK would also obfuscate any unique metal features.

• I use HL/C to compile for my M1. I also run HL under rosetta and it runs really well. If I need to do deep debugging I run it on my PC. It's not ideal, but I would rather use my Mac than switch to PC in general.

[ncannasse]

Hi Ryan, Would definitely love to include Metal support in Heaps. In theory the design should be quite similar to what we did with DirectX12. You can also look at the very WIP WebGPU branch that follows similar design. Don't hesitate if you have any questions, we can put you in touch with one of Shiro engine dev to get you answers. Best, Nicolas

Le mer. 20 mars 2024 à 07:30, Ryan Cleven @.***> a écrit :

Hey, thanks for the thoughts!

-

Originally, I was going to use MoltenVK, but I've been warned against it by some professional engine writers that have used it before. Also, there's a lot of benefits to directly using Metal, without having a second layer to try to figure out why something isn't working. Metal has subtly different paradigms from Vulcan which can make the second layer more complicated than It needs to be. Using MoltenVK would also obfuscate any unique metal features.

I use HL/C to compile for my M1. I also run HL under rosetta and it runs really well. If I need to do deep debugging I run it on my PC. It's not ideal, but I would rather use my Mac than switch to PC in general.

— Reply to this email directly, view it on GitHub https://github.com/HeapsIO/heaps/issues/1200#issuecomment-2009711917, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHZXQHPDXQCOYKSR3JI6R3YZGMQ5AVCNFSM6AAAAABE6YBH32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBZG4YTCOJRG4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[onehundredfeet]

Great. I was planning on basing it on the DX12 integration. It looks a lot more complete than the WebGPU one.

[Nazariglez]

This is interesting, why not use dawn or wgpu so we keep shaders in WGSL and backends in Vulkan, Metal, and Dx all at once?

[Nazariglez]

hey @onehundredfeet sorry for hijack this thread a bit, related to this:

I use HL/C to compile for my M1. I also run HL under rosetta and it runs really well. If I need to do deep debugging I run it on my PC. It's not ideal, but I would rather use my Mac than switch to PC in general.

I am using rosetta to run it HashLink too, and it runs well. Although I am not able to compile using HL/C. Always end up with:

ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Using: gcc -O3 -o myapp -std=c11 -I out out/main.c -lhl -L/usr/local/lib -arch arm64. Can I ask you to share your steps or the command you use to compile on arm macs, please?

[onehundredfeet]

You need to compile the hl library as either arm or universal.

[onehundredfeet]

typically the installed HL lib in /usr/local/lib is going to be intel.

You'll have to build hashlink yourself. I have a modified makefile that builds arm. Here's my build script.

#!/bin/sh

make clean
arch -x86_64 make -j 32
mv libhl.dylib libhl_x86_64.dylib
mv fmt.hdll fmt_x86_64.dylib
mv openal.hdll openal_x86_64.dylib
mv sdl.hdll sdl_x86_64.dylib
mv ssl.hdll ssl_x86_64.dylib
mv ui.hdll ui_x86_64.dylib
mv uv.hdll uv_x86_64.dylib
mv hl hl_x86_64
mv libhl.a libhl_x86_64.a

make clean
make -j 32
mv libhl.dylib libhl_arm64.dylib
mv fmt.hdll fmt_arm64.dylib
mv openal.hdll openal_arm64.dylib
mv sdl.hdll sdl_arm64.dylib
mv ssl.hdll ssl_arm64.dylib
mv ui.hdll ui_arm64.dylib
mv uv.hdll uv_arm64.dylib
mv hl hl_arm64
mv libhl.a libhl_arm64.a

lipo -create -output libhl.dylib libhl_x86_64.dylib libhl_arm64.dylib 
lipo -create -output fmt.hdll fmt_x86_64.dylib fmt_arm64.dylib
lipo -create -output openal.hdll openal_x86_64.dylib openal_arm64.dylib
lipo -create -output sdl.hdll sdl_x86_64.dylib sdl_arm64.dylib
lipo -create -output ssl.hdll ssl_x86_64.dylib ssl_arm64.dylib
lipo -create -output ui.hdll ui_x86_64.dylib ui_arm64.dylib
lipo -create -output uv.hdll uv_x86_64.dylib uv_arm64.dylib
lipo -create -output libhl.a libhl_x86_64.a libhl_arm64.a
mv hl_x86_64 hl

Note, there's a static library in there as well that I've added. You'll have to remove it to use the build script.

[Nazariglez]

Thanks! I will try it. However, I may be wrong but this steps are to install HL right? If that's the case I got it working with brew here https://github.com/deepnight/gameBase/issues/109#issuecomment-2118409098.

The issue I have is building the output on the c folder using gcc. (As I said, I need to test what you shared, sorry if I misunderstanding it.)

[onehundredfeet]

I rebuild hashlink myself. My process is not trivial.

1) Modify makefile to be able to build x64 or arm 2) build using the above script 3) make install

My process is about making a universal version of the hl.dylib and all the hdll's. There will only be an x86 version of the hl executable. I can only use the arm version of the hl.dylib and hdll's when using hl/c build, which I've also customized.

I may write this up somewhere, but currently, it's not for the faint of heart.