Build universal native libraries on macOS [2 XMR]

[woodser]

Haveno can use native libraries instead of monero-wallet-rpc for wallet functionality using the monero-java project.

Currently, the native libraries built for macOS are platform-dependent, so they must be built for x86_64 and ARM64.

This issue requests making the native libraries universal on macOS, so multiple builds are not necessary.

This issue shares a bounty with the related issue: https://github.com/woodser/monero-java/issues/81

[github-actions[bot]]

There is a bounty on this issue, the amount is in the title. The reward will be awarded to the first person or group of people who resolves this issue.

If you are starting to work on this bounty, please write a comment, so that we can assign the issue to you. We expect contributors to provide a PR in a reasonable time frame or, in case of an extensive work, updates on their progresses. We will unassign the issue if we feel the assignee is not responsive or has abandoned the task.

Read the full conditions and details of our bounty system.

[preland]

I’ll look into this

[woodser]

Excellent, happy to have your help. Thanks.

[niyid]

I'm curious; is the goal to avoid multiple builds or to have a universal native library on Mac?

[woodser]

Universal native library for macOS x86_64 and ARM64.

[niyid]

Universal native library for macOS x86_64 and ARM64.

What if the universal native library is generated from already built x86_64 and ARM64 libraries? Then that involves multiple builds. That is why I ask.

[woodser]

That could be one way to do it, but I was expecting to use cmake configuration and toolchains like monero-project, to avoid multiple builds and be able to do it from one machine: https://github.com/monero-project/monero/tree/master/cmake

[niyid]

For clarity; is it allowed to use the platform libraries to generate this universal library?

On Tue, Apr 2, 2024, 3:45 AM woodser @.***> wrote:

Universal native library for macOS x86_64 and ARM64.

— Reply to this email directly, view it on GitHub https://github.com/haveno-dex/haveno/issues/843#issuecomment-2030974944, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2MWRYLTIRJRT54RPG3YNDY3ILWPAVCNFSM6AAAAABFJMPZMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQHE3TIOJUGQ . You are receiving this because you commented.Message ID: @.***>

[niyid]

I think 2 builds to generate the 2 architecture libraries might still be necessary. And then the final step will be to combine both into the universal library.

On Tue, Apr 2, 2024, 1:12 PM Oyeniyi Dada @.***> wrote:

For clarity; is it allowed to use the platform libraries to generate this universal library?

On Tue, Apr 2, 2024, 3:45 AM woodser @.***> wrote:

Universal native library for macOS x86_64 and ARM64.

— Reply to this email directly, view it on GitHub https://github.com/haveno-dex/haveno/issues/843#issuecomment-2030974944, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2MWRYLTIRJRT54RPG3YNDY3ILWPAVCNFSM6AAAAABFJMPZMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQHE3TIOJUGQ . You are receiving this because you commented.Message ID: @.***>

[woodser]

I'm reading that multiple flags can be passed to the build command, e.g. CXXFLAGS="-arch x86_64 -arch arm64 -mmacosx-version-min=11.0 -std=c++11"

But I haven't looked into it deeply.

[niyid]

Are you already using set(CMAKE_OSX_ARCHITECTURES "x86_64;arm64")?

On Tue, Apr 2, 2024, 1:49 PM woodser @.***> wrote:

I'm reading that multiple flags can be passed to the build command, e.g. CXXFLAGS="-arch x86_64 -arch arm64 -mmacosx-version-min=11.0 -std=c++11"

But I haven't looked into it deeply.

— Reply to this email directly, view it on GitHub https://github.com/haveno-dex/haveno/issues/843#issuecomment-2031957315, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2MWR5TG55OBPQ6T7A46BLY3KSNXAVCNFSM6AAAAABFJMPZMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZRHE2TOMZRGU . You are receiving this because you commented.Message ID: @.***>

[woodser]

Nope, but sounds worth trying.

[niyid]

I believe set(CMAKE_OSX_ARCHITECTURES "x86_64;arm64") should be used; followed by lipo to combine the 2 arch libraries into a universal library.

On Tue, Apr 2, 2024, 2:01 PM Oyeniyi Dada @.***> wrote:

Are you already using set(CMAKE_OSX_ARCHITECTURES "x86_64;arm64")?

On Tue, Apr 2, 2024, 1:49 PM woodser @.***> wrote:

I'm reading that multiple flags can be passed to the build command, e.g. CXXFLAGS="-arch x86_64 -arch arm64 -mmacosx-version-min=11.0 -std=c++11"

But I haven't looked into it deeply.

— Reply to this email directly, view it on GitHub https://github.com/haveno-dex/haveno/issues/843#issuecomment-2031957315, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2MWR5TG55OBPQ6T7A46BLY3KSNXAVCNFSM6AAAAABFJMPZMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZRHE2TOMZRGU . You are receiving this because you commented.Message ID: @.***>

[niyid]

Any progress on this? I will be taking a look.

On Tue, Apr 2, 2024, 2:05 PM Oyeniyi Dada @.***> wrote:

I believe set(CMAKE_OSX_ARCHITECTURES "x86_64;arm64") should be used; followed by lipo to combine the 2 arch libraries into a universal library.

On Tue, Apr 2, 2024, 2:01 PM Oyeniyi Dada @.***> wrote:

Are you already using set(CMAKE_OSX_ARCHITECTURES "x86_64;arm64")?

On Tue, Apr 2, 2024, 1:49 PM woodser @.***> wrote:

I'm reading that multiple flags can be passed to the build command, e.g. CXXFLAGS="-arch x86_64 -arch arm64 -mmacosx-version-min=11.0 -std=c++11"

But I haven't looked into it deeply.

— Reply to this email directly, view it on GitHub https://github.com/haveno-dex/haveno/issues/843#issuecomment-2031957315, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2MWR5TG55OBPQ6T7A46BLY3KSNXAVCNFSM6AAAAABFJMPZMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZRHE2TOMZRGU . You are receiving this because you commented.Message ID: @.***>

[woodser]

set(CMAKE_OSX_ARCHITECTURES "x86_64;arm64") did not work by adding it to CMakeLists.txt in monero-cpp and monero-java; the resulting libs work on x86_64 but not arm64.

[niyid]

...and you added the lipo command?

I have not tested this yet (no Mac):

cmake_minimum_required(VERSION 3.4.1)

project(monero-java-jni)

# Set architectures for Apple platforms
if (APPLE)
    set(ARCHITECTURES x86_64 arm64)
endif()

# For other platforms, use the original script
if (WIN32)
    add_definitions( "-D_GLIBCXX_USE_NANOSLEEP=1" )
    add_definitions( "-DWIN32_LEAN_AND_MEAN" )
    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wa,-mbig-obj -O2 -fPIC -std=c++14 -F/Library/Frameworks -pthread -lcrypto -lcrypt32") 
else()
    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -std=c++14 -F/Library/Frameworks -pthread")
endif()

#SET(CMAKE_C_COMPILER /path/to/c/compiler)
#SET(CMAKE_CXX_COMPILER /path/to/cpp/compiler)

#############
# System
#############

set(MONERO_CPP "${CMAKE_SOURCE_DIR}/external/monero-cpp")
message(STATUS MONERO_CPP : ${MONERO_CPP} : ${MONERO_CPP})

set(MONERO_CPP_SRC "${MONERO_CPP}/src")
set(MONERO_PROJECT ${MONERO_CPP}/external/monero-project)
set(MONERO_PROJECT_SRC "${MONERO_PROJECT}/src")

# java header includes
if(NOT DEFINED ENV{JAVA_HOME} OR "$ENV{JAVA_HOME}" STREQUAL "")
  message(FATAL_ERROR "JAVA_HOME variable not set, for example: export JAVA_HOME=/path/to/jdk")
else()
  include_directories("$ENV{JAVA_HOME}")
  include_directories("$ENV{JAVA_HOME}/include")
  if (APPLE)
    include_directories("$ENV{JAVA_HOME}/include/darwin")
  elseif (WIN32)
    include_directories("$ENV{JAVA_HOME}/include/win32")
  else()
    include_directories("$ENV{JAVA_HOME}/include/linux")
  endif()
endif()

# monero-project header includes
include_directories("${MONERO_CPP}/external/libsodium/include/sodium")
include_directories("${MONERO_CPP}/external/openssl-sdk/include")
include_directories("${MONERO_CPP_SRC}/")
include_directories("${MONERO_PROJECT}/contrib/epee/include")
include_directories("${MONERO_PROJECT}/external/")
include_directories("${MONERO_PROJECT}/external/easylogging++")
include_directories("${MONERO_PROJECT}/external/rapidjson/include")
include_directories("${MONERO_PROJECT_SRC}/")
include_directories("${MONERO_PROJECT_SRC}/crypto")
include_directories("${MONERO_PROJECT_SRC}/crypto/crypto_ops_builder/include/")
include_directories("${MONERO_PROJECT_SRC}/wallet")
include_directories("${MONERO_PROJECT_SRC}/wallet/api")

# TODO: remove TRUEs, how are APPLE, DEPENDS, etc initialized?
if (TRUE OR HIDAPI_FOUND OR LibUSB_COMPILE_TEST_PASSED)
  if (APPLE)
    if(TRUE OR DEPENDS)
      list(APPEND EXTRA_LIBRARIES "-framework Foundation -framework IOKit -framework AppKit")
    else()
      find_library(COREFOUNDATION CoreFoundation)
      find_library(IOKIT IOKit)
      find_library(APPKIT AppKit)
      list(APPEND EXTRA_LIBRARIES ${IOKIT})
      list(APPEND EXTRA_LIBRARIES ${COREFOUNDATION})
      list(APPEND EXTRA_LIBRARIES ${APPKIT})
    endif()
  endif()
  if (WIN32)
    list(APPEND EXTRA_LIBRARIES setupapi)
  endif()
endif()

message(STATUS EXTRA_LIBRARIES: ${EXTRA_LIBRARIES})

############
# Boost
############

set(Boost_NO_BOOST_CMAKE 1)
set(Boost_USE_MULTITHREADED ON)
find_package(Boost 1.58 QUIET REQUIRED COMPONENTS chrono date_time filesystem program_options regex serialization wserialization system thread)
message(STATUS "Using Boost include dir at ${Boost_INCLUDE_DIR}")
include_directories(${Boost_INCLUDE_DIR})

############
# OpenSSL
############

if (APPLE AND NOT IOS)
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=default -std=c++14")
  if (NOT OPENSSL_ROOT_DIR)
      EXECUTE_PROCESS(COMMAND brew --prefix openssl
        OUTPUT_VARIABLE OPENSSL_ROOT_DIR
        OUTPUT_STRIP_TRAILING_WHITESPACE)
    message(STATUS "Using OpenSSL found at ${OPENSSL_ROOT_DIR}")
  endif()
endif()

find_package(OpenSSL REQUIRED)
message(STATUS "Using OpenSSL include dir at ${OPENSSL_INCLUDE_DIR}")
include_directories(${OPENSSL_INCLUDE_DIR})

if(STATIC AND NOT IOS)
  if(UNIX)
    set(OPENSSL_LIBRARIES "${OPENSSL_LIBRARIES};${CMAKE_DL_LIBS};${CMAKE_THREAD_LIBS_INIT}")
  endif()
endif()

if (WIN32)
  list(APPEND OPENSSL_LIBRARIES ws2_32 crypt32)
endif()

######################
# monero-cpp
######################

add_library(monero-cpp SHARED IMPORTED)

# import shared c++ library
if (APPLE)
  set_target_properties(monero-cpp PROPERTIES IMPORTED_LOCATION ./libmonero-cpp.dylib)
elseif (WIN32)
  set_target_properties(monero-cpp PROPERTIES IMPORTED_LOCATION ./libmonero-cpp.dll)
  set_target_properties(monero-cpp PROPERTIES IMPORTED_IMPLIB ./libmonero-cpp.dll.a)
else()
  set_target_properties(monero-cpp PROPERTIES IMPORTED_LOCATION ./libmonero-cpp.so)
endif()

# Check if building for macOS to combine builds into a universal library using lipo
if(APPLE)
    # Add flags and configurations for each architecture
    foreach(arch IN ITEMS ${ARCHITECTURES})
        set(CMAKE_C_FLAGS_${arch} "${CMAKE_C_FLAGS} -arch ${arch}")
        set(CMAKE_CXX_FLAGS_${arch} "${CMAKE_CXX_FLAGS} -arch ${arch}")
        set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_${arch} ${CMAKE_BINARY_DIR}/${arch})
        add_library(monero-java-${arch} SHARED ${MONERO_JNI_SRC_FILES})
        target_link_libraries(monero-java-${arch}
            monero-cpp
            ${Boost_LIBRARIES}
            ${OPENSSL_LIBRARIES}
            ${EXTRA_LIBRARIES}
        )
    endforeach()

    add_custom_command(TARGET monero-java-x86_64 monero-java-arm64
        POST_BUILD COMMAND
        lipo -create
        $<TARGET_FILE:monero-java-x86_64>
        $<TARGET_FILE:monero-java-arm64>
        -output ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/libmonero-java.dylib
    )

    # Install the combined library
    INSTALL(TARGETS monero-java-x86_64
        RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT Runtime
        LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT Runtime
        ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT Development
    )
endif()

[NamesCode]

I can test this for you, I have an M2 mac and I can emulate an x86-64 one using UTM

[preland]

I can test this for you, I have an M2 mac and I can emulate an x86-64 one using UTM

The main issue (sorry, I kinda dropped the ball on this issue, was working on other things) was/is getting monero to properly static compile for both x86_64 and arm64 on the same system. I was able to get that far in GitHub workflows, but I am still unable to properly test if lipo will create a proper universal binary (especially the cctools Linux port, which seems to hate GitHub workflows with a burning passion)

the most recent workflow run I did is here; if you could run lipo on the artifacts and see if it works properly, that would be highly appreciated. It should, but honestly it wouldn’t shock me at this point if it found a way to mess things up

[NamesCode]

I can test this for you, I have an M2 mac and I can emulate an x86-64 one using UTM

The main issue (sorry, I kinda dropped the ball on this issue, was working on other things) was/is getting monero to properly static compile for both x86_64 and arm64 on the same system. I was able to get that far in GitHub workflows, but I am still unable to properly test if lipo will create a proper universal binary (especially the cctools Linux port, which seems to hate GitHub workflows with a burning passion)

the most recent workflow run I did is here; if you could run lipo on the artifacts and see if it works properly, that would be highly appreciated. It should, but honestly it wouldn’t shock me at this point if it found a way to mess things up

Heyo! Good news!! after running lipo create ~/Downloads/Cross-Mac-x86_64/{Binary name here} ~/Downloads/Cross-Mac-aarch64/{Binary name here} -output ./{Binary name here} and chmod +x'ing the resulting binaries both worked perfectly on aarch64. No warnings or issues. Had a bit of a surprise when monerod started syncing too

The binaries themselves work in practice so you may be able to just run the above command in the final linker step? I havent taken a look at the build.yml myself to see where it is erroring.

Proof it works too ^^:

[niyid]

@ all, thank you all for your contribution and proving the idea works.

[NamesCode]

If there's anyway I can help further I'm more than happy too ^^

I'll see if I can take a look at the GitHub workflows tomorrow if you'd like @preland unless it's not going to be used in the final build?

[preland]

If there's anyway I can help further I'm more than happy too ^^

I'll see if I can take a look at the GitHub workflows tomorrow if you'd like @preland unless it's not going to be used in the final build?

That would be appreciated; once that is working properly, then the path to getting this issue resolved will be as “simple” as monero->monero-cpp->monero-java->haveno

[NamesCode]

If there's anyway I can help further I'm more than happy too ^^ I'll see if I can take a look at the GitHub workflows tomorrow if you'd like @preland unless it's not going to be used in the final build?

That would be appreciated; once that is working properly, then the path to getting this issue resolved will be as “simple” as monero->monero-cpp->monero-java->haveno

It seems the issue is that the build-universal-macos-app action requires that the artifacts be pushed to a release first before it can create them.

The problem with this is Monero uses gitian for releases which includes signings, hashes etc. This means the fatbinary action needs gitian to be done first before it itself can be pushed into that release, making a chicken or the egg problem.

The two ideas that come to mind at the moment is either:

A)

Haveno bundles the fat binaries itself, using the github action your using to pull moneros official down and merge them. A benefit of this is its generally easier and simpler to do, however a major downide of this though is that it is not officially signed by monero. For most people this isn't an issue as no one is really checking the file signature and we can sign it ourselves as if they are using haveno, they most likely already trust the binaries we provide.

B)

We hand roll a custom lipo maker thats able to use the previous artifacts somehow, possible a bash snippet post macos build in a macos env or creating a custom program and dockerizing it. A benefit of this is it can be merged into monero upstream but there are quite a few more cons as, it may be more convoluted to pull the artifacts and lipo them and we would have to hand roll a program just to do so if we dont go the bash way which requires a macos vm. Im not too well versed in github actions however so take this option with a pinch of salt since I dont fully know all the options available with it, we may just be able to pass the artifacts directly to amalgam (The name of the program the universal-macos-releaser uses under the hood) and have it produce a fat binary which gets uploaded to artifacts and included in the gitian release. If this is the case then this'd be ideal.

---

With my limited knowledge I'd want to say option A may be better but if we can just pull an artifact and put it into amalgam on the monero upstream then option B would be the best case.

I'll take a look into option B after the 18th since I have exams over the next few weeks making me unable to work on things.

[preland]

Thanks for looking into it :)

Part of the issue is that, with the current dependency chain, the best solution would be to create the brand new lipo target within the main monero repo, and then have the dependency get fixed that way.

In terms of getting that done: we can figure out how to get the action working, and then wait for the PR to get merged; in the meantime we would just directly use the PR commit for monero-cpp

I noticed that there was a single pre-existing GitHub workflow that would use lipo to create a universal binary, but it wasn’t working when I tried it out. Maybe we could make a custom one? Tbh if it has to rely on a MacOS runner that shouldn’t be too much of a concern at this point (since the lipo step isn’t really in scope for Monero static builds, as it is rly little more than packaging. At most we may need to add some extra information in build documentation)

[preland]

Ignore my previous comment; it is wrong.

Looking into it deeply, the easiest way to do this is to build the entire libraries for both x86 and Arm separately, then combine using lipo.

I currently have PRs open in monero-cpp and monero-java that do just that. I can confirm that it works on a Debian 12 system. I have yet to test it on a macOS host (Windows won't work out of the box currently as I didn't update portions of the toolchain to allow for it to be used).

My only concern currently is whether or not the built artifact in monero-java will currently be accessible via jitpack.io (ie what we use to access gradle/maven/whatever artifacts from repositories). If that concern is addressed, then this will be a fully functioning universal macOS build pipeline (that can be compiled statically on other platforms).

One current pitfall is that the bash scripts will build the entire project from scratch. I can start looking into other changes once necessary.