Issues with C# invocation on MacOS

[Gekctek]

I have been dealing with an issue of using this library for the last few days with MacOs and im stuck. I have followed your c# example code and others (like https://github.com/NethermindEth/cortex-cryptography-bls) and I get everything to work just fine on Windows and Linux, just not Macs. The problem comes when trying to load the binary itself

 System.DllNotFoundException : Unable to load shared library 'bls384_256' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libbls384_256, 0x0001): tried: 'libbls384_256' (no such file), '/usr/local/lib/libbls384_256' (no such file), '/usr/lib/libbls384_256' (no such file), '/Users/runner/work/ICP.NET/ICP.NET/test/Candid.Tests/bin/Debug/net6.0/libbls384_256' (no such file)

I set DYLD_PRINT_LIBRARIES to 1 but I dont see any output, so that is not helpful. I have played with renaming it and placing it in different folders and confirming the file is there without any luck. I have even tried doing a more manual approach with directly calling dlopen, with the same result

 dlerror(); // Clear any existing error
libraryHandle = dlopen(libraryName, RtldNow);
if (libraryHandle == IntPtr.Zero)
{
    string error = dlerror();
    if (!string.IsNullOrEmpty(error))
    {
        throw new Exception($"Error loading native library: {error}");
    }
}

I was doing some research and found that the library can fail to load if one or more of its dependecies are not there. So i ran otool -L libbls384_256.dylib and I get

 lib/libbls384_256.dylib (compatibility version 0.0.0, current version 0.0.0)
    /opt/homebrew/opt/gmp/lib/libgmp.10.dylib (compatibility version 15.0.0, current version 15.1.0)
    /opt/homebrew/opt/gmp/lib/libgmpxx.4.dylib (compatibility version 11.0.0, current version 11.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1200.3.0)

It seems like GMP is installed but technically in /usr/local/opt/gmp instead of /opt/homebrew/opt/gmp. Im not sure if thats a problem

The cherry on top is that I dont have a mac so I have been working with other people/using github actions to do testing which just slows everything down.

My issues might be out of scope for this specific project and are more Mac related but I do have specific questions that maybe you can help with: 1) I see there is If the option MCL_USE_GMP=0 (resp.MCL_USE_OPENSSL=0) is used then GMP (resp. OpenSSL) is not used. When should these be used? Could I just set them to 0 and have no issues? are there tradeoffs? 2) Do you have mac users? Is there anything i should look out for? 3) Literally any help because Im at my breaking point lol

[herumi]

I see there is If the option MCL_USE_GMP=0 (resp.MCL_USE_OPENSSL=0) is used then GMP (resp. OpenSSL) is not used. When should these be used? Could I just set them to 0 and have no issues? are there tradeoffs?

The current mcl does not use GMP/OpenSSL, so you can disable them by MCL_USE_GMP=0 MCL_USE_OPENSSL=0.

[herumi]

Could you try make -f Makefile.onelib and use bls/lib/darwin/arm64/libbls384_256.a?

[herumi]

Makefile.onelib generates a static library, so I'll modify it to generate a shared library.

[herumi]

I've updated it at dev branch. Could you try make -f Makefile.onlib BLS_LIB_SHARED=1 or make -f Makefile.onelib BLS_LIB_SHARED=1 ETH_CFLAGS=-DBLS_ETH?

[Gekctek]

@herumi Will give this a try. Tried out the MCL_USE_GMP=0 MCL_USE_OPENSSL=0 but no change with that

[herumi]

I tried the following steps on Intel Mac, and then it ran well.

cd bls
make -f Makefile.onelib BLS_LIB_SHARED=1 LIB_DIR=bin -j
open ffi/cs/bls.sln and build a release bls.exe.
cd ./bin
mono ../ffi/cs/obj/x64/Release/bls.exe

I'll try it on M1 mac.

[herumi]

It ran well on M1 mac.

bls% arch
arm64
bls% mono --version
Mono JIT compiler version 6.12.0.182 (tarball Mon Jan 16 21:39:07 UTC 2023)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:
        SIGSEGV:       normal
        Notification:  kqueue
        Architecture:  arm64
        Disabled:      none
        Misc:          softdebug
        Interpreter:   yes
        LLVM:          supported, not enabled.
        Suspend:       preemptive
        GC:            sgen
bls% mono bin/bls.exe
curveType=0
TestId
TestSecretKey
sec.SetByCSPRNG=19acafbd1743bb015b3c1c25d6c98dd4bdb9b08e805120dd617cdd0c19e8b118
...(snip)..
all tests succeed

[Gekctek]

HMMMMM So we tried that, but everytime we build the lib ourselves i get the System.DllNotFoundException : Unable to load shared library 'bls384_256' or one of its dependencies.

BUT I copied the built DLL from another project https://www.nuget.org/packages/Cortex.Cryptography.Bls https://github.com/NethermindEth/cortex-cryptography-bls And surprisingly it DOES load the dll BUT it has BLS_ETH=1 which we dont want, so it doesn't work, but it loads

im really not sure whats going on with this T attached a zip of the one that does load but is BLS_ETH and the one that doesnt load but is configured properly. I dont know if that will help, but maybe you could try running it? idk I appreciate your help on this libbls384_256.dylib.zip

[herumi]

I will ask you some basic questions.

• Is your Mac Intel or M1?
• Did you run https://github.com/herumi/bls/issues/102#issuecomment-1517255403, and does it run well?
• DLL should be in the same directory as the exe file.

[Gekctek]

1) Great question, im not sure, ive been outsourcing my mac build to another person. Ill ask them. 2) Yes, Ive done that one and a bunch of others with different parameters. All have the same issue 3) Ive eliminated this as an issue as I can load the DLL from the nuget package https://www.nuget.org/packages/Cortex.Cryptography.Bls but it is configured with the BLS_ETH=1, but we need =0

If I dont have someone else test the dll, I am using a github action with the macos-latest (v12) and thats got to be an intel processor

Are you able to send me the Intel dylib you made above? Maybe its just a M1 thing and all this headache was for nothing. Really should just get a mac myself lol

[herumi]

This is the Intel dylib built without BLS_ETH. intel-mac.tgz

[Gekctek]

This is the Intel dylib built without BLS_ETH. intel-mac.tgz

Perfect. I think that was the issue. Had to get a little more creative with the code to support multiple architectures and OSes but I think i got it

Thanks again for your help

[Gekctek]

Run into a new issue. Apparently new macos versions are requiring binaries to be signed by developers? Do you have any experience with this or have run into this issue? I feel like I'm going to end up having to just buy a mac but was curious if you had any knowledge around this Link i found https://www.kencochrane.com/2020/08/01/build-and-sign-golang-binaries-for-macos-with-github-actions/

Otherwise i might be looking into wasm maybe? I see you have a wasm option for the browser but I've seen that C# can run wasm, so that might be a direction i go

[Gekctek]

As far as the WASM build, is it possible to build the standalone WASM file like emcc ... -o output.wasm omits generating either JavaScript or HTML launcher file, and produces a single Wasm file built in standalone mode as if the -sSTANDALONE_WASM settting had been used. I wasn't having much success poking at it, but im looking to just have the raw WASM that I can utilize in C# vs in the browser.

[herumi]

I have not used macOS code signing. According to https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution , there are a lot of procedures.

If you are looking for the WASM version, see https://github.com/herumi/bls-wasm . Also, these questions are on a different topic than the original issue, so please create a new issue.

[Gekctek]

Good call, ill make a new topic.