daphne-eu / daphne

DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines
Apache License 2.0
67 stars 59 forks source link

MAC OSX support #759

Open flashboss opened 5 months ago

flashboss commented 5 months ago

I'm on Mac OSX 23.5.0 Darwin Kernel Version 23.5.0 arm64

I would compile the last Daphne version but When I do ,/build.sh I get this error:

checking for architecture... arm64 checking for OS... darwin checking for OS version... 23.5.0 checking for perf_event workaround level... autodetect checking for if MIC should be used... no checking for if NEC should be used... no checking for xlc... no checking for icc... no checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for xlf... no checking for ifort... no checking for gfortran... gfortran checking whether we are using the GNU Fortran 77 compiler... yes checking whether gfortran accepts -g... yes checking for mpicc... no checking for gawk... no checking for mawk... no checking for nawk... no checking for awk... awk checking how to run the C preprocessor... gcc -E checking whether ln -s works... yes checking whether make sets $(MAKE)... yes checking for ranlib... ranlib checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for ANSI C header files... rm: conftest.dSYM: is a directory rm: conftest.dSYM: is a directory yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking minix/config.h usability... no checking minix/config.h presence... no checking for minix/config.h... no checking whether it is safe to define EXTENSIONS... yes checking for ANSI C header files... (cached) yes checking for inline... inline checking whether time.h and sys/time.h may both be included... yes checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking c_asm.h usability... no checking c_asm.h presence... no checking for c_asm.h... no checking intrinsics.h usability... no checking intrinsics.h presence... no checking for intrinsics.h... no checking mach/mach_time.h usability... yes checking mach/mach_time.h presence... yes checking for mach/mach_time.h... yes checking sched.h usability... yes checking sched.h presence... yes checking for sched.h... yes checking for gethrtime... no checking for read_real_time... no checking for time_base_to_time... no checking for clock_gettime... yes checking for mach_absolute_time... yes checking for sched_getcpu... no checking for timercreate and timerettime symbols in base system... not found checking for timercreate and timerettime symbols in -lrt... not found checking for timercreate and timerettime symbols in -lrt -lpthread... configure: error: cannot find timercreate and timerettime symbols neither in the base system libraries nor in -lrt, nor in -lrt -lpthread

The problem is papi that need get_time symbols but MAC OSX doesn't support them. Here the Papi thred: https://github.com/icl-utk-edu/papi/issues/190

What is the right way to compile Daphne?

pdamme commented 5 months ago

Thanks for creating this issue, @flashboss. So far, we haven't designed the DAPHNE build workflow to support Mac OSX, since we mainly target Linux platforms (especially Ubuntu as described in the GettingStarted guide). Thus, DAPHNE does not build on Mac OSX natively at the moment. However, there are two possible solutions:

  1. Our general approach to make DAPHNE run on platforms where a native build is not supported [yet] is using pre-built container images. You can find instructions on using these containers in our quickstart guide for users or for developers. Feel free to let us know in case you encounter any problems with those containers.
  2. We are already working on making DAPHNE build natively on Mac OSX, but so far with low priority. As far as I know, @philipportner has made some progress in this direction recently. He might be able to help you best. He is currently out of the office, but I think he should be able to join the discussion beginning of next week. In the meantime, maybe I can help you (even though I don't have a Mac to test with). The concrete problem you are currently facing is that PAPI doesn't support Mac OSX. PAPI is only required for certain extra features of DAPHNE (related to profiling). It should not be difficult to make PAPI an optional dependency in DAPHNE; I will push a commit for that tomorrow and let you know.
philipportner commented 5 months ago

Hi @flashboss ,

As @pdamme already mentioned, DAPHNE is currently only supported through docker containers on macOS. To do so, please pull from main and start the docker container through ./containers/run-docker-example.sh. Once inside you should be able to build DAPHNE via ./build.sh.

pdamme commented 5 months ago

Thanks for the quickfix to make the docker containers work on Mac OSX, @philipportner.

pdamme commented 5 months ago

@flashboss Using the docker containers is most likely the easiest way to get DAPHNE running on Mac OSX. In case you prefer a native build without docker: I have pushed a commit to main, which enables building DAPHNE without PAPI (and, consequently, without profiling support). Feel free to try ./build.sh --no-papi or ./test.sh --no-papi on your Mac OSX system. We're happy to help if you encounter any further obstacles.

flashboss commented 4 months ago

Hi, build through container now is ok. Building without container I get this error:

lucastancapiano@MacBook-Pro-7 daphne % ./build.sh --no-papi not using PAPI [DAPHNE].................................................................................................................................................................................[20.06.24 09:50:54] [DAPHNE]...Welcome to....................................................................................................................................................................[20.06.24 09:50:54] [DAPHNE].................................................................................................................................................................................[20.06.24 09:50:54] [DAPHNE]...._....................................................................................................................................................................[20.06.24 09:50:54] [DAPHNE]...| .....................| ............................................................................................................................................[20.06.24 09:50:54] [DAPHNE]...| €€€€€€€\ ____ __ | €€___ __....................................................................................................................[20.06.24 09:50:54] [DAPHNE]...| €€ | €€ | \ / \ | €€ \ | \ / ...................................................................................................................[20.06.24 09:50:54] [DAPHNE]...| €€ | €€ \€€€€€€| €€€€€€| €€€€€€€| €€€€€€€| €€€€€€..................................................................................................................[20.06.24 09:50:54] [DAPHNE]...| €€ | €€ / €€| €€ | €€| €€ | €€| €€ | €€| €€ €€..................................................................................................................[20.06.24 09:50:54] [DAPHNE]...| €€/ €€| €€€€€€€| €€/ €€| €€ | €€| €€ | €€| €€€€€€€€..................................................................................................................[20.06.24 09:50:54] [DAPHNE]...| €€ €€ \€€ €€| €€ €€| €€ | €€| €€ | €€ \€€ ..................................................................................................................[20.06.24 09:50:54] [DAPHNE]... \€€€€€€€ \€€€€€€€| €€€€€€€ \€€ \€€ \€€ \€€ \€€€€€€€..................................................................................................................[20.06.24 09:50:54] [DAPHNE].......................| €€......................................................................................................................................................[20.06.24 09:50:54] [DAPHNE].......................| €€......................................................................................................................................................[20.06.24 09:50:54] [DAPHNE]........................\€€..................EU-H2020.//.957407..................................................................................................................[20.06.24 09:50:54] [DAPHNE].................................................................................................................................................................................[20.06.24 09:50:54]

[DAPHNE]..No need to build hwloc again...................................................................................................................................................[20.06.24 09:50:55] [DAPHNE]..Get Antlr version 4.9.2........................................................................................................................................................[20.06.24 09:50:55] [DAPHNE]..Download Antlr v4.9.2 java archive.............................................................................................................................................[20.06.24 09:50:55] [DAPHNE]..Download Antlr v4.9.2 Runtime..................................................................................................................................................[20.06.24 09:51:33] mkdir: illegal option -- - usage: mkdir [-pv] [-m mode] directory_name ...

auge commented 4 months ago

mkdir on macos does not support '--parents' only the short form '-p'

flashboss commented 4 months ago

I know it would be more correct to use the container but I think it is right to use -p instead of -parents as it is universal and guarantees greater portability

pdamme commented 4 months ago

@flashboss Great to hear that you could make DAPHNE work in the container on Mac OS, so you can already start using DAPHNE. Thanks to @auge, the build script uses mkdir -p now. Feel free to try again.

flashboss commented 4 months ago

I have this error now:

ninja: build stopped: subcommand failed. [DAPHNE]..Applying 0001-antlr-gtest-silence-warnings.patch...............................................................................................................................[22.06.24 08:08:36] patch: **** can't cd to /Users/lucastancapiano/git/daphne/thirdparty/build/antlr4-cpp-runtime-4.9.2-source/runtime/thirdparty/utfcpp/extern/gtest/: No such file or directory [2/308] Performing download step (git clone) for 'utfcpp' FAILED: runtime/utfcpp-prefix/src/utfcpp-stamp/utfcpp-download cd /Users/lucastancapiano/git/daphne/thirdparty/build/antlr4-cpp-runtime-4.9.2-source/runtime/thirdparty && /usr/local/Cellar/cmake/3.20.0/bin/cmake -P /Users/lucastancapiano/git/daphne/thirdparty/build/antlr4-cpp-runtime-4.9.2-source/runtime/utfcpp-prefix/tmp/utfcpp-gitclone.cmake && /usr/local/Cellar/cmake/3.20.0/bin/cmake -E touch /Users/lucastancapiano/git/daphne/thirdparty/build/antlr4-cpp-runtime-4.9.2-source/runtime/utfcpp-prefix/src/utfcpp-stamp/utfcpp-download Cloning into 'utfcpp'... fatal: unable to connect to github.com: github.com[0: 140.82.121.3]: errno=Operation timed out

Cloning into 'utfcpp'... fatal: unable to connect to github.com: github.com[0: 140.82.121.4]: errno=Operation timed out

Cloning into 'utfcpp'... fatal: unable to connect to github.com: github.com[0: 140.82.121.4]: errno=Operation timed out

-- Had to git clone more than once: 3 times. CMake Error at /Users/lucastancapiano/git/daphne/thirdparty/build/antlr4-cpp-runtime-4.9.2-source/runtime/utfcpp-prefix/tmp/utfcpp-gitclone.cmake:31 (message): Failed to clone repository: 'git://github.com/nemtrif/utfcpp'

ninja: build stopped: subcommand failed.

philipportner commented 4 months ago

Hi @flashboss , @auge and @pdamme , thanks for the feedback so far. Be assured that I am actively working on supporting DAPHNE on macOS and already have solved some of these problems. I will update you on this issue once compiling and running DAPHNE works on macOS @flashboss !

flashboss commented 4 months ago

Are there news? Actually I have this error:

build.sh --no-papi

*** Assembler checking dependency style of gcc... gcc3 checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking for fgrep... /usr/bin/grep -F checking for atomic builtin atomics... yes checking for atomic_compare_exchange_n... yes checking if __atomic_compare_exchange_n() gives correct results... yes checking if __int128 atomic compare-and-swap is always lock-free... yes checking if .proc/endp is needed... no checking directive for setting text section... .text checking directive for exporting symbols... .globl checking for objdump... objdump checking if .note.GNU-stack is needed... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump: error: 'conftest.o': Invalid/Unsupported object file format no checking suffix for labels... : checking prefix for global symbol labels... none configure: error: Could not determine global symbol label prefix

philipportner commented 4 months ago

Hi @flashboss , I'm working on it but I still have quite some troubles to get it working on macOS. I will create a branch and draft PR once I have something that runs, even if it may have limited functionality. Until then, please use the docker image.