building static libraries with shared library wrapper

[nsec1]

Ref. #74 haveno-dex/haveno#795

Now, there is only monero-java.so. It wraps monero, monero-cpp, boost etc. Working with haveno on linux (--useNativeXmrWallet=true). Depends woodser/monero-cpp#64

[woodser]

I've applied this PR and https://github.com/woodser/monero-cpp/pull/64.

Looks promising, but I'm having trouble building it.

On Ubuntu 20.04, I first get this error:

[100%] Built target monero-cpp
-- MONERO_CPP:/home/woodser/git/monero-java/external/monero-cpp
-- EXTRA_LIBRARIES:
-- Using Boost include dir at /home/woodser/git/monero-java/external/boost/boost
-- Found boost library: boost_chrono;boost_date_time;boost_filesystem;boost_program_options;boost_regex;boost_serialization;boost_wserialization;boost_system;boost_thread
-- Using OpenSSL include dir at /usr/include
-- Using OpenSSL libs: /usr/lib/aarch64-linux-gnu/libssl.a;/usr/lib/aarch64-linux-gnu/libcrypto.a;-lpthread;dl;dl;-lpthread
-- Using libsodium library at /usr/lib/aarch64-linux-gnu/libsodium.so
-- Using monero-project build: /home/woodser/git/monero-java/external/monero-cpp/external/monero-project/build/release
-- Found libunbound library: /usr/local/lib/libunbound.a
-- Configuring done
-- Generating done
-- Build files have been written to: /home/woodser/git/monero-java/build
make[2]: *** No rule to make target 'libmonero-cpp.a', needed by 'libmonero-java.so'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/monero-java.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

Setting BUILD_SHARED_LIBS to OFF in monero-cpp and deleting the build directories creates the needed static monero-cpp.a.

Then the build fails during linking:

[ 50%] Building CXX object CMakeFiles/monero-java.dir/src/main/cpp/monero_jni_bridge.cpp.o
[100%] Linking CXX shared library libmonero-java.so
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(sha1-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/lib/aarch64-linux-gnu/libcrypto.a(sha1-armv8.o): in function `sha1_block_armv8':
(.text+0x1240): dangerous relocation: unsupported relocation
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(chacha-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/lib/aarch64-linux-gnu/libcrypto.a(chacha-armv8.o):(.text+0x20): dangerous relocation: unsupported relocation
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): relocation R_AARCH64_ADR_PREL_LO21 against symbol `poly1305_blocks' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): in function `poly1305_init':
(.text+0x40): dangerous relocation: unsupported relocation
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): relocation R_AARCH64_ADR_PREL_LO21 against symbol `poly1305_emit' which may bind externally can not be used when making a shared object; recompile with -fPIC
(.text+0x48): dangerous relocation: unsupported relocation
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/lib/aarch64-linux-gnu/libcrypto.a(poly1305-armv8.o): in function `poly1305_emit_neon':
(.text+0x9a0): dangerous relocation: unsupported relocation
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(sha256-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/lib/aarch64-linux-gnu/libcrypto.a(sha256-armv8.o): in function `sha256_block_data_order':
(.text+0xf88): dangerous relocation: unsupported relocation
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libcrypto.a(sha512-armv8.o): relocation R_AARCH64_PREL64 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/lib/aarch64-linux-gnu/libcrypto.a(sha512-armv8.o): in function `sha512_block_data_order':
(.text+0x1108): dangerous relocation: unsupported relocation
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/monero-java.dir/build.make:122: libmonero-java.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/monero-java.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

On macOS, I get this error:

[ 50%] Building CXX object CMakeFiles/monero-java.dir/src/main/cpp/monero_jni_bridge.cpp.o
In file included from /Users/woodser/git/monero-java/src/main/cpp/monero_jni_bridge.cpp:39:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_full.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_model.h:55:
/Users/woodser/git/monero-java/external/monero-cpp/src/daemon/monero_daemon_model.h:55:10: fatal error: 'boost/property_tree/ptree.hpp' file not found
#include <boost/property_tree/ptree.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Changing set(Boost_INCLUDE_DIR "${BOOST_PATH}/boost") to set(Boost_INCLUDE_DIR "${BOOST_PATH}") seems to get further, but fails on the error:

In file included from /Users/woodser/git/monero-java/src/main/cpp/monero_jni_bridge.cpp:39:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_full.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_model.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/daemon/monero_daemon_model.h:55:
In file included from /Users/woodser/git/monero-java/external/boost/boost/property_tree/ptree.hpp:21:
In file included from /Users/woodser/git/monero-java/external/boost/boost/multi_index_container.hpp:25:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/at.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/at_impl.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/advance.hpp:19:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/negate.hpp:17:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/integral_c.hpp:32:
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for this enumeration type [-Wenum-constexpr-conversion]
    typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
                              ^
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
#   define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
                                              ^
In file included from /Users/woodser/git/monero-java/src/main/cpp/monero_jni_bridge.cpp:39:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_full.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_model.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/daemon/monero_daemon_model.h:55:
In file included from /Users/woodser/git/monero-java/external/boost/boost/property_tree/ptree.hpp:21:
In file included from /Users/woodser/git/monero-java/external/boost/boost/multi_index_container.hpp:25:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/at.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/at_impl.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/advance.hpp:19:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/negate.hpp:17:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/integral_c.hpp:32:
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for this enumeration type [-Wenum-constexpr-conversion]
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
#   define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)

[woodser]

Hoping the errors can be resolved because it would be really great to build to portable, static libraries. :)

[nsec1]

Setting BUILD_SHARED_LIBS to OFF in monero-cpp and deleting the build directories creates the needed static monero-cpp.a.

This is done in bin/build_libmonero_java.sh

Then the build fails during linking:

OpenSSL will have to become a module built with -fPIC. I will look into this.

Changing set(Boost_INCLUDE_DIR "${BOOST_PATH}/boost") to set(Boost_INCLUDE_DIR "${BOOST_PATH}") seems to get further, but fails on the error:

I will look into this

[nsec1]

Then the build fails during linking:

@woodser , could you try to build again on Ubuntu 20.04 aarch64-linux-gnu?

[woodser]

I was able to build successfully on Ubuntu 20.04 aarch64-linux-gnu, but I'm still having trouble building on macOS, first failing on this error:

[ 50%] Building CXX object CMakeFiles/monero-java.dir/src/main/cpp/monero_jni_bridge.cpp.o
In file included from /Users/woodser/git/monero-java/src/main/cpp/monero_jni_bridge.cpp:39:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_full.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_model.h:55:
/Users/woodser/git/monero-java/external/monero-cpp/src/daemon/monero_daemon_model.h:55:10: fatal error: 'boost/property_tree/ptree.hpp' file not found
#include <boost/property_tree/ptree.hpp>

I'm confused why the build should work at all, including on Ubuntu, because set(Boost_INCLUDE_DIR "${BOOST_PATH}/boost") would use the full include path of ./external/boost/boost/property_tree/ptree.hpp, which doesn't exist.

By modifying CMakeLists.txt to set(Boost_INCLUDE_DIR "${BOOST_PATH}"), it's able to find the include path to the property tree, but then later fails on this error:

In file included from /Users/woodser/git/monero-java/src/main/cpp/monero_jni_bridge.cpp:39:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_full.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_model.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/daemon/monero_daemon_model.h:55:
In file included from /Users/woodser/git/monero-java/external/boost/boost/property_tree/ptree.hpp:21:
In file included from /Users/woodser/git/monero-java/external/boost/boost/multi_index_container.hpp:25:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/at.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/at_impl.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/advance.hpp:19:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/negate.hpp:17:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/integral_c.hpp:32:
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for this enumeration type [-Wenum-constexpr-conversion]
    typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
                              ^
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
#   define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
                                              ^
In file included from /Users/woodser/git/monero-java/src/main/cpp/monero_jni_bridge.cpp:39:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_full.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/wallet/monero_wallet_model.h:55:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/daemon/monero_daemon_model.h:55:
In file included from /Users/woodser/git/monero-java/external/boost/boost/property_tree/ptree.hpp:21:
In file included from /Users/woodser/git/monero-java/external/boost/boost/multi_index_container.hpp:25:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/at.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/at_impl.hpp:18:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/advance.hpp:19:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/negate.hpp:17:
In file included from /Users/woodser/git/monero-java/external/boost/boost/mpl/integral_c.hpp:32:
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for this enumeration type [-Wenum-constexpr-conversion]
/Users/woodser/git/monero-java/external/boost/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
#   define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)

[woodser]

I see that monero-cpp is using boost and openssl with these paths instead of the new submodules, wondering if that could be a problem:

-- Using Boost include dir at /opt/homebrew/include
-- Using OpenSSL found at /opt/homebrew/opt/openssl@3
-- Using OpenSSL include dir at /opt/homebrew/opt/openssl@3/include
-- Using libsodium library at /opt/homebrew/lib/libsodium.dylib
-- Using HIDAPI include dir at /opt/homebrew/include/hidapi

Perhaps boost and openssl should be included within ./external/monero-cpp/external instead and used to build monero-cpp as well?

[nsec1]

I'm confused why the build should work at all, including on Ubuntu, because set(Boost_INCLUDE_DIR "${BOOST_PATH}/boost") would use the full include path of ./external/boost/boost/property_tree/ptree.hpp, which doesn't exist.

It works because you have boost installed. You are write, I have fixed boost include path.

[nsec1]

I see that monero-cpp is using boost and openssl with these paths instead of the new submodules, wondering if that could be a problem:

Whats the boost version installed on macOS? monero-cpp and monero-project require boost >= 1.58. monero-java, monero-cpp and monero-project should be built with compatible versions of boost. I chose to use boost 1.71 because Ubuntu 20 uses this version. I think yours macoS is using boost < 1.71. If its true, We can force monero-cpp to require 1.71.

[woodser]

Whats the boost version installed on macOS?

I have boost 1.84 installed through brew.

monero-java, monero-cpp and monero-project should be built with compatible versions of boost

Wondering again then if boost and openssl should be a submodule within external/monero-cpp/external, which both monero-java and monero-cpp build from?

This would be consistent with how monero-ts works, where boost [1] and openssl [2] are downloaded to external/monero-cpp/external.

[nsec1]

Wondering again then if boost and openssl should be a submodule within external/monero-cpp/external, which both monero-java and monero-cpp build from?

Ok. I will try. But monero-project also use boost and openssl and maybe we dont have control of it.

[woodser]

Ok. I will try.

Maybe it's not necessary, and the boost versions merely need adjusted to be compatible, but I would think monero-cpp should be built with the same version as monero-java.

But monero-project also use boost and openssl and maybe we dont have control of it.

Don't know if the -fPIC difference could be a problem, but I'm not so well versed in this build stuff.

[nsec1]

@woodser could you try to build on macos again? You have to apply the patch woodser/monero-cpp#64

[woodser]

Hm, I still end up with these errors building monero-java with the new changes to this and monero-cpp:

In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/utils/gen_utils.cpp:53:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/utils/gen_utils.h:57:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/lexical_cast.hpp:32:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/lexical_cast/try_lexical_convert.hpp:43:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/lexical_cast/detail/converter_numeric.hpp:36:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/cast.hpp:33:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/converter.hpp:13:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/conversion_traits.hpp:13:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/detail/conversion_traits.hpp:18:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/detail/int_float_mixture.hpp:19:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/mpl/integral_c.hpp:32:
/Users/woodser/git/monero-java/build/install/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for this enumeration type [-Wenum-constexpr-conversion]
    typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
                              ^
/Users/woodser/git/monero-java/build/install/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
#   define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
                                              ^
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/utils/gen_utils.cpp:53:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/src/utils/gen_utils.h:57:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/lexical_cast.hpp:32:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/lexical_cast/try_lexical_convert.hpp:43:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/lexical_cast/detail/converter_numeric.hpp:36:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/cast.hpp:33:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/converter.hpp:13:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/conversion_traits.hpp:13:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/detail/conversion_traits.hpp:18:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/numeric/conversion/detail/int_float_mixture.hpp:19:
In file included from /Users/woodser/git/monero-java/build/install/include/boost/mpl/integral_c.hpp:32:
/Users/woodser/git/monero-java/build/install/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for this enumeration type [-Wenum-constexpr-conversion]
/Users/woodser/git/monero-java/build/install/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
#   define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)

I confirmed that the build succeeds from the master branches.

[nsec1]

Could you past here or send me the full build output? I want to know if build is using boost lib from submodule or system.

[woodser]

Could you past here or send me the full build output? I want to know if build is using boost lib from submodule or system.

build.log

[nsec1]

@woodser could you try again on macos and put the log here please? You have to apply the patch https://github.com/woodser/monero-cpp/pull/64

[woodser]

It's failing to build boost with the latest changes to monero-java and monero-cpp:

build.log

[woodser]

This is the error:

./boost/unordered/detail/fwd.hpp:53:16: error: no member named 'piecewise_construct' in namespace 'std'; did you mean 'piecewise_construct_t'?
    using std::piecewise_construct;
          ~~~~~^~~~~~~~~~~~~~~~~~~
               piecewise_construct_t

[nsec1]

Its related with the newer macOS plus boost 1.7x. See https://github.com/bambulab/BambuStudio/issues/3957 I will try to use 1.8x. Anyway, I need to emulate macos env on linux. Any suggestion?

[woodser]

Yeah I saw that. I just updated my OS to the latest version, but it didn't help.

Using 1.8x sounds good, it's what I've been using in the libraries anyway.

I don't have any experience emulating mac in linux unfortunately.

[nsec1]

Build failed on ubuntu-20.04 with boost 1.86. We need to update monero-project. See https://github.com/monero-project/monero/pull/9313 Can I do this in https://github.com/woodser/monero-cpp/pull/64?

[woodser]

Ok, I'm in the process of updating to monero-project v0.18.3.4 now.

[woodser]

Ok it's all updated to monero-project v0.18.3.4.

[nsec1]

@woodser , build ok on ubuntu-20.04. Could you try again on macos?

[woodser]

Sure, please resolve the merge conflict with master. :)

[woodser]

With the PRs applied to monero-java and monero-cpp (and the merge conflict resolved), and using the latest monero-project v0.18.3.4 (pushed to my master at https://github.com/woodser/monero), I get these errors on building monero-project:

[ 44%] Building CXX object contrib/epee/src/CMakeFiles/obj_epee.dir/int-util.cpp.o
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/cryptonote_basic/cryptonote_format_utils_basic.cpp:31:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/cryptonote_basic/cryptonote_format_utils.h:33:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/cryptonote_basic/cryptonote_basic_impl.h:33:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/cryptonote_basic/cryptonote_basic.h:44:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/serialization/crypto.h:37:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/crypto/chacha.h:44:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/crypto/hash.h:37:
/Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/crypto/generic-ops.h:36:10: fatal error: 'sodium/crypto_verify_32.h' file not found
#include <sodium/crypto_verify_32.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 46%] Building CXX object contrib/epee/src/CMakeFiles/obj_epee.dir/portable_storage.cpp.o
1 error generated.
make[3]: *** [src/device/CMakeFiles/obj_device.dir/device.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
[ 46%] Building CXX object contrib/epee/src/CMakeFiles/obj_epee.dir/misc_language.cpp.o
[ 46%] Building CXX object src/cryptonote_basic/CMakeFiles/obj_cryptonote_basic.dir/cryptonote_basic_impl.cpp.o
1 error generated.
[100%] Built target generate_translations_header
make[3]: *** [src/device_trezor/CMakeFiles/obj_device_trezor.dir/device_trezor.cpp.o] Error 1
make[2]: *** [src/device_trezor/CMakeFiles/obj_device_trezor.dir/all] Error 2
[ 46%] Building CXX object src/blockchain_db/CMakeFiles/obj_blockchain_db.dir/lmdb/db_lmdb.cpp.o
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/ringct/rctSigs.cpp:35:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/common/util.h:48:
In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/crypto/hash.h:37:
/Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/crypto/generic-ops.h:36:10: fatal error: 'sodium/crypto_verify_32.h' file not found
#include <sodium/crypto_verify_32.h>

Here's the full build log:

build.log

monero-project succeeds to build if I run make release-static directly within the repo, so it must be related to building the full stack from ./bin/build_libmonero_java.sh.

I confirmed that the build succeeds by switching to the latest master branch of all 3 projects. To ensure the build is completely fresh, I first deleted the build directory of all 3 projects.

[woodser]

If I first build monero-project with make release-static directly in the repo, the build gets pretty far, only failing on the final linking step, due to undefined references to boost, which points to the fact that different versions of boost are being used. Feels like we're getting closer.

[nsec1]

Sure, please resolve the merge conflict with master. :)

I'm confused. There is no conflict. I've already resolved them.

[woodser]

I observed a merge conflict with this PR on this line, which changed in the last release: https://github.com/woodser/monero-java/blob/c0bd817662b23b924c203db87f1b455cc00547f5/CMakeLists.txt#L86

You should see GitHub indicates this message:

I think you just need to pull the latest master and git rebase master on your branch, then force push.

[nsec1]

I resolved this conflict here: 4cb116abbf1a30070f841ae638c11ebc872af096. See image, its the msg: "This branch has no conflicts with the base branch"

[woodser]

Hm, maybe squash your commits?

[nsec1]

The question is why github show different msgs?

[nsec1]

Hm, maybe squash your commits?

is it a problem?

[woodser]

The question is why github show different msgs?

Yeah it's strange. I don't know why, but GitHub won't even let me click the button to merge. Only thing I can think to do is squash your commits, which is worth a try.

[nsec1]

The question is why github show different msgs?

Yeah it's strange. I don't know why, but GitHub won't even let me click the button to merge. Only thing I can think to do is squash your commits, which is worth a try.

ok, squash it then

[woodser]

You would need to squash it, since it's not ready to squash and merge yet :)

[nsec1]

I did a merge commit to resolve the conflict.

The question is why github show different msgs?

Maybe "Allow merge commits" is disable in project config

[woodser]

You should be able to simply git reset --soft HEAD~9, git commit -m "your commit message", and then git push --force to squash to a single commit.

[nsec1]

You should be able to simply git reset --soft HEAD~9, git commit -m "your commit message", and then git push --force to squash to a single commit.

Not exactly this because it has a conflict but done

[nsec1]

If I first build monero-project with make release-static directly in the repo, the build gets pretty far, only failing on the final linking step, due to undefined references to boost, which points to the fact that different versions of boost are being used. Feels like we're getting closer.

This is how I test build:

# Make sure cloned branch is clean:
git clone -b sharedLibraryWrapper https://github.com/nsec1/monero-java.git
cd monero-java

# Update modules:
git submodule update --init --recursive

# Apply woodser/monero-cpp#64 patch
(cd external/monero-cpp && curl https://patch-diff.githubusercontent.com/raw/woodser/monero-cpp/pull/64.diff | patch -p1)

# buid all
./bin/build_libmonero_java.sh

I would like to see "Using OpenSSL libs: ..." and "Using Boost libs: ..." values on monero-cpp build output.

[woodser]

I applied your latest PR. Still getting the error I referenced above: fatal error: 'sodium/crypto_verify_32.h' file not found.

The error occurs while building monero-project, so it never gets to building monero-cpp and monero-java (and never prints "Using OpenSSL libs".

Here's the full log: build.log

[nsec1]

do you have libsodium (with header files) installed?

[woodser]

I have libsodium with header files placed within monero-cpp/external, which is referenced within CMakeLists.txt in both monero-cpp and monero-java. The build is succeeding with this on master.

[woodser]

Building monero-project standalone works. Building monero-cpp and monero-project with ./bin/build_libmonero_cpp.sh also works.

Then building monero-java fails on the linking step with these errors (appears to be a boost mismatch?):

[100%] Linking CXX shared library libmonero-java.dylib
Undefined symbols for architecture arm64:
  "boost::filesystem::detail::weakly_canonical(boost::filesystem::path const&, boost::filesystem::path const&, boost::system::error_code*)", referenced from:
      tools::wallet2::store_to(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, epee::wipeable_string const&, bool) in libwallet.a[2](wallet2.cpp.o)
  "boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator&, boost::filesystem::path const&, unsigned int, boost::filesystem::detail::directory_iterator_params*, boost::system::error_code*)", referenced from:
      std::__1::__function::__func<mlog_configure(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, bool, unsigned long, unsigned long)::$_0, std::__1::allocator<mlog_configure(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, bool, unsigned long, unsigned long)::$_0>, void (char const*, unsigned long)>::operator()(char const*&&, unsigned long&&) in libepee.a[7](mlog.cpp.o)
  "boost::filesystem::detail::canonical(boost::filesystem::path const&, boost::filesystem::path const&, boost::system::error_code*)", referenced from:
      tools::wallet2::store_to(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, epee::wipeable_string const&, bool) in libwallet.a[2](wallet2.cpp.o)
  "boost::filesystem::detail::copy_file(boost::filesystem::path const&, boost::filesystem::path const&, unsigned int, boost::system::error_code*)", referenced from:
      tools::copy_file(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) in libcommon.a[8](util.cpp.o)
  "_hid_close", referenced from:
      hw::io::device_io_hid::disconnect() in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_enumerate", referenced from:
      hw::io::device_io_hid::connect(unsigned int, unsigned int, boost::optional<int>, boost::optional<unsigned short>) in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_error", referenced from:
      hw::io::safe_hid_error(hid_device_*) in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_free_enumeration", referenced from:
      hw::io::device_io_hid::connect(unsigned int, unsigned int, boost::optional<int>, boost::optional<unsigned short>) in libdevice.a[6](device_io_hid.cpp.o)
      hw::io::device_io_hid::connect(unsigned int, unsigned int, boost::optional<int>, boost::optional<unsigned short>) in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_init", referenced from:
      hw::io::device_io_hid::init() in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_open_path", referenced from:
      hw::io::device_io_hid::connect(unsigned int, unsigned int, boost::optional<int>, boost::optional<unsigned short>) in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_read", referenced from:
      hw::io::device_io_hid::exchange(unsigned char*, unsigned int, unsigned char*, unsigned int, bool) in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_read_timeout", referenced from:
      hw::io::device_io_hid::exchange(unsigned char*, unsigned int, unsigned char*, unsigned int, bool) in libdevice.a[6](device_io_hid.cpp.o)
      hw::io::device_io_hid::exchange(unsigned char*, unsigned int, unsigned char*, unsigned int, bool) in libdevice.a[6](device_io_hid.cpp.o)
  "_hid_write", referenced from:
      hw::io::device_io_hid::exchange(unsigned char*, unsigned int, unsigned char*, unsigned int, bool) in libdevice.a[6](device_io_hid.cpp.o)
ld: symbol(s) not found for architecture arm64

monero-cpp prints that it's using Boost and OpenSSL:

-- Using Boost include dir at /opt/homebrew/include
-- Using Boost lib version: 1_84
-- Using Boost libs: /opt/homebrew/lib/libboost_chrono-mt.dylib;/opt/homebrew/lib/libboost_date_time-mt.dylib;/opt/homebrew/lib/libboost_filesystem-mt.dylib;/opt/homebrew/lib/libboost_program_options-mt.dylib;/opt/homebrew/lib/libboost_regex-mt.dylib;/opt/homebrew/lib/libboost_serialization-mt.dylib;/opt/homebrew/lib/libboost_wserialization-mt.dylib;/opt/homebrew/lib/libboost_system-mt.dylib;/opt/homebrew/lib/libboost_thread-mt.dylib;/opt/homebrew/lib/libboost_atomic-mt.dylib
-- Using OpenSSL found at /opt/homebrew/opt/openssl@3
-- Using OpenSSL include dir at /opt/homebrew/opt/openssl@3/include
-- Using OpenSSL lib version: 
-- Using OpenSSL libs: /opt/homebrew/opt/openssl@3/lib/libssl.dylib;/opt/homebrew/opt/openssl@3/lib/libcrypto.dylib
-- Using libsodium library at /opt/homebrew/lib/libsodium.dylib
-- Using HIDAPI include dir at /opt/homebrew/include/hidapi
-- Using monero-project build: /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/build/release

monero-java prints that it's using Boost and OpenSSL:

-- Using Boost include dir at /Users/woodser/git/monero-java/build/install/include
-- Using Boost lib version: 1_86
-- Using Boost libs: /Users/woodser/git/monero-java/build/install/lib/libboost_chrono.a;/Users/woodser/git/monero-java/build/install/lib/libboost_date_time.a;/Users/woodser/git/monero-java/build/install/lib/libboost_filesystem.a;/Users/woodser/git/monero-java/build/install/lib/libboost_program_options.a;/Users/woodser/git/monero-java/build/install/lib/libboost_regex.a;/Users/woodser/git/monero-java/build/install/lib/libboost_serialization.a;/Users/woodser/git/monero-java/build/install/lib/libboost_wserialization.a;/Users/woodser/git/monero-java/build/install/lib/libboost_system.a;/Users/woodser/git/monero-java/build/install/lib/libboost_thread.a;/Users/woodser/git/monero-java/build/install/lib/libboost_atomic.a
-- Found OpenSSL: /Users/woodser/git/monero-java/build/install/lib/libcrypto.a (found version "3.3.1")  
-- Using OpenSSL include dir at /Users/woodser/git/monero-java/build/install/include
-- Using OpenSSL lib version: 
-- Using OpenSSL libs: /Users/woodser/git/monero-java/build/install/lib/libssl.a;/Users/woodser/git/monero-java/build/install/lib/libcrypto.a
-- Using libsodium library at /opt/homebrew/lib/libsodium.dylib
-- Using monero-project build: /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/build/release
-- Found libunbound library: /usr/local/lib/libunbound.a

Here's the full log for that: build.log

Deleting the build directories from monero-java, monero-cpp, and monero-project and rebuilding from the top-level monero-java yields the error: sodium/crypto_verify_32.h within monero-project, which only resolves by deleting its build directory and building standalone again.

[woodser]

It does appear that monero-java and monero-cpp are using different locations for Boost and OpenSSL, which could explain a mismatch?

[nsec1]

Yes, it could. CMAKE_PREFIX_PATH env var is not being exported to cmake executions in /bin/build_libmonero_cpp.sh on macos. It works on linux. I will try to force to pass this by parameter to cmake.

[nsec1]

Deleting the build directories from monero-java, monero-cpp, and monero-project and rebuilding from the top-level monero-java yields the error: sodium/crypto_verify_32.h within monero-project, which only resolves by deleting its build directory and building standalone again.

How do you build standalone?

Try to build like this:

    (cd build/release && \
     cmake -D STATIC=ON -D BUILD_64=ON -D CMAKE_BUILD_TYPE=Release\
           -D OPENSSL_ROOT_DIR=/Users/woodser/git/monero-java/build/install -D CMAKE_PREFIX_PATH=/Users/woodser/git/monero-java/build/install ../.. && \
     make -j$HOST_NCORES wallet)

[woodser]

Thanks for your continued effort on this.

Getting an error with the latest changes while building monero-project:

In file included from /Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/cryptonote_basic/connection_context.cpp:33:
/Users/woodser/git/monero-java/external/monero-cpp/external/monero-project/src/p2p/p2p_protocol_defs.h:195:7: error: static assertion failed due to requirement 'std::is_pod<boost::uuids::uuid>::value': t_type must be a POD type.
      KV_SERIALIZE_VAL_POD_AS_BLOB(network_id)

Here's the full build log: build.log

How do you build standalone?

I can build monero-project standalone using make release-static directly within the monero-project directory.

I get the same error as above in monero-project by running your recommended command:

    (cd build/release && \
     cmake -D STATIC=ON -D BUILD_64=ON -D CMAKE_BUILD_TYPE=Release\
           -D OPENSSL_ROOT_DIR=/Users/woodser/git/monero-java/build/install -D CMAKE_PREFIX_PATH=/Users/woodser/git/monero-java/build/install ../.. && \
     make -j$HOST_NCORES wallet)

It does feel like we're close since the build has succeeded on Ubuntu, but I wonder if macOS emulation would help get us past this next step to ensure it builds there too.

[nsec1]

Ok. I am looking into darling to emulate macos.

[woodser]

I realized my last error was due to an error building monero-project v0.18.4.3 (see https://github.com/monero-project/monero/pull/9450), but after fixing that, I'm back to the earlier error:

'sodium/crypto_verify_32.h' file not found