Apple M1 import error: an incompatible architecture (have 'x86_64', need 'arm64e')

yiershi commented 2 years ago

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

import dgl
error message: OSError: dlopen(/Users/haizhi/miniforge3/lib/python3.9/site-packages/dgl-0.8-py3.9-macosx-11.0-arm64.egg/dgl/libdgl.dylib, 0x0006): tried: '/Users/haizhi/miniforge3/lib/python3.9/site-packages/dgl-0.8-py3.9-macosx-11.0-arm64.egg/dgl/libdgl.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/libdgl.dylib' (no such file), '/usr/lib/libdgl.dylib' (no such file)

Expected behavior

Environment

OS (e.g., Linux): Mac OS 12.1 (Monterey)
How you installed DGL: source
Build command you used (if compiling from source): https://docs.dgl.ai/install/index.html#macos mkdir build cd build cmake -DUSE_OPENMP=off -DCMAKE_C_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DCMAKE_CXX_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DUSE_AVX=OFF -DUSE_LIBXSMM=OFF .. make -j4 cd ../python python setup.py install
Python version: 3.9

VoVAllen commented 2 years ago

It's weird. Did you see the libdgl.dylib in the build directory?

yiershi commented 2 years ago

yes, the libdgl.dylib is in the build directory.

$ locate libdgl.dylib /Users/haizhi/PycharmProjects/git项目/dgl/build/libdgl.dylib /Users/haizhi/miniforge3/lib/python3.9/site-packages/dgl-0.8-py3.9-macosx-11.0-arm64.egg/dgl/libdgl.dylib

$ ll /Users/haizhi/PycharmProjects/git项目/dgl/build/libdgl.dylib -rwxr-xr-x 1 haizhi staff 8.8M 2 17 18:31 /Users/haizhi/PycharmProjects/git项目/dgl/build/libdgl.dylib

VoVAllen commented 2 years ago

The error message is saying DGL is built on x86 instead of arm arch. (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')

We don't have Apple M1 so far. Could you check your build environment?

yiershi commented 2 years ago

The error message is saying DGL is built on x86 instead of arm arch. (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')

We don't have Apple M1 so far. Could you check your build environment?

$ arch arm64

$ python --version Python 3.9.7

$ system_profiler SPHardwareDataType Hardware:

Hardware Overview:

  Model Name: MacBook Pro
  Model Identifier: MacBookPro18,3
  Chip: Apple M1 Pro
  Total Number of Cores: 10 (8 performance and 2 efficiency)
  Memory: 32 GB
  System Firmware Version: 7429.61.2
  OS Loader Version: 7429.61.2
  …… …… ……

VoVAllen commented 2 years ago

Could you post the full cmake logs?

yiershi commented 2 years ago

$ cmake -DUSE_OPENMP=off -DCMAKE_C_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DCMAKE_CXX_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DUSE_AVX=OFF -DUSE_LIBXSMM=OFF ..

-- The C compiler identification is AppleClang 13.0.0.13000029
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Start configuring project dgl
-- Performing Test SUPPORT_CXX14
-- Performing Test SUPPORT_CXX14 - Success
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
-- Looking for fopen64
-- Looking for fopen64 - not found
-- Looking for C++ include cxxabi.h
-- Looking for C++ include cxxabi.h - found
-- Looking for nanosleep
-- Looking for nanosleep - found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include  
-- /Users/haizhi/PycharmProjects/git项目/dgl/third_party/dmlc-core/cmake/build_config.h.in -> include/dmlc/build_config.h
-- Performing Test SUPPORT_MSSE2
-- Performing Test SUPPORT_MSSE2 - Success
-- Looking for execinfo.h
-- Looking for execinfo.h - found
-- Looking for getline
-- Looking for getline - found
-- Performing Test UV_LINT_W4
-- Performing Test UV_LINT_W4 - Failed
-- Performing Test UV_LINT_NO_UNUSED_PARAMETER_MSVC
-- Performing Test UV_LINT_NO_UNUSED_PARAMETER_MSVC - Failed
-- Performing Test UV_LINT_NO_CONDITIONAL_CONSTANT_MSVC
-- Performing Test UV_LINT_NO_CONDITIONAL_CONSTANT_MSVC - Failed
-- Performing Test UV_LINT_NO_NONSTANDARD_MSVC
-- Performing Test UV_LINT_NO_NONSTANDARD_MSVC - Failed
-- Performing Test UV_LINT_NO_NONSTANDARD_EMPTY_TU_MSVC
-- Performing Test UV_LINT_NO_NONSTANDARD_EMPTY_TU_MSVC - Failed
-- Performing Test UV_LINT_NO_NONSTANDARD_FILE_SCOPE_MSVC
-- Performing Test UV_LINT_NO_NONSTANDARD_FILE_SCOPE_MSVC - Failed
-- Performing Test UV_LINT_NO_NONSTANDARD_NONSTATIC_DLIMPORT_MSVC
-- Performing Test UV_LINT_NO_NONSTANDARD_NONSTATIC_DLIMPORT_MSVC - Failed
-- Performing Test UV_LINT_NO_HIDES_LOCAL
-- Performing Test UV_LINT_NO_HIDES_LOCAL - Failed
-- Performing Test UV_LINT_NO_HIDES_PARAM
-- Performing Test UV_LINT_NO_HIDES_PARAM - Failed
-- Performing Test UV_LINT_NO_HIDES_GLOBAL
-- Performing Test UV_LINT_NO_HIDES_GLOBAL - Failed
-- Performing Test UV_LINT_NO_CONDITIONAL_ASSIGNMENT_MSVC
-- Performing Test UV_LINT_NO_CONDITIONAL_ASSIGNMENT_MSVC - Failed
-- Performing Test UV_LINT_NO_UNSAFE_MSVC
-- Performing Test UV_LINT_NO_UNSAFE_MSVC - Failed
-- Performing Test UV_LINT_WALL
-- Performing Test UV_LINT_WALL - Success
-- Performing Test UV_LINT_NO_UNUSED_PARAMETER
-- Performing Test UV_LINT_NO_UNUSED_PARAMETER - Success
-- Performing Test UV_LINT_STRICT_PROTOTYPES
-- Performing Test UV_LINT_STRICT_PROTOTYPES - Success
-- Performing Test UV_LINT_EXTRA
-- Performing Test UV_LINT_EXTRA - Success
-- Performing Test UV_LINT_UTF8_MSVC
-- Performing Test UV_LINT_UTF8_MSVC - Failed
-- Performing Test UV_F_STRICT_ALIASING
-- Performing Test UV_F_STRICT_ALIASING - Success
-- summary of build options:
    Install prefix:  /usr/local
    Target system:   Darwin
    Compiler:
      C compiler:    /Library/Developer/CommandLineTools/usr/bin/cc
      CFLAGS:         -O2 -Wall -fPIC -DXBYAK_DONT_USE_MAP_JIT -DIDXTYPEWIDTH=64 -DREALTYPEWIDTH=32 -DLINUX -D_FILE_OFFSET_BITS=64 -DNDEBUG -DNDEBUG2 -DHAVE_EXECINFO_H -DHAVE_GETLINE -O3

-- Found uv: 1.38.1 (found version "1.38.1") 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/haizhi/PycharmProjects/git项目/dgl/build

VoVAllen commented 2 years ago

Could you try cmake -DCMAKE_OSX_ARCHITECTURES=arm64 -DUSE_OPENMP=off -DCMAKE_C_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DCMAKE_CXX_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DUSE_AVX=OFF -DUSE_LIBXSMM=OFF .. and rebuild the DGL?

yiershi commented 2 years ago

Could you try cmake -DCMAKE_OSX_ARCHITECTURES=arm64 -DUSE_OPENMP=off -DCMAKE_C_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DCMAKE_CXX_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DUSE_AVX=OFF -DUSE_LIBXSMM=OFF .. and rebuild the DGL?

3ks. The problem sovled according to your solution.

$ python -c 'import dgl; print(dgl.__version__)'
[20:53:03] /Users/haizhi/PycharmProjects/git项目/dgl/src/runtime/tensordispatch.cc:43: TensorDispatcher: dlopen failed: dlopen(/Users/haizhi/miniforge3/lib/python3.9/site-packages/dgl-0.8-py3.9-macosx-11.0-arm64.egg/dgl/tensoradapter/pytorch/libtensoradapter_pytorch_1.10.0.dylib, 0x0001): tried: '/Users/haizhi/miniforge3/lib/python3.9/site-packages/dgl-0.8-py3.9-macosx-11.0-arm64.egg/dgl/tensoradapter/pytorch/libtensoradapter_pytorch_1.10.0.dylib' (no such file), '/usr/local/lib/libtensoradapter_pytorch_1.10.0.dylib' (no such file), '/usr/lib/libtensoradapter_pytorch_1.10.0.dylib' (no such file)
0.8

somya-15 commented 2 years ago

I am getting the same error saying- The error message is saying Error in py_initialize(config$python, config$libpython, config$pythonhome, : (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')

And I am using- Model Name: MacBook Air Chip: Apple M1 Air Memory: 16 GB

I am unable to understand how to solve this error.

VoVAllen commented 2 years ago

@somya-15 Could you try the compile command I posted above?

edc-ff commented 2 years ago

Hi all. I'm having the same issue and running the above command results in: CMake Error: The source directory "/Users/myuser" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI.

I'm new to MacOS and I'm not sure how to proceed.

BubblyOrca commented 2 years ago

I am getting a similar error while I am trying to import an external Python package within jupyter notebook:

(mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/lib/cygrpc.cpython-310-darwin.so' (no such file)

edc-ff commented 2 years ago

Solved my problem. It had to do with the numpy version that was specified in a project.toml file, which was earlier than 1.21.4. Afaik, this is the first numpy version that supports Apple's M1.

However, this wasn't enough to fix the issue. But installing Cython in the same environment solved it.

Haimonti commented 1 year ago

I run into the following problem:

cmake -DUSE_OPENMP=off -DCMAKE_C_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DCMAKE_CXX_FLAGS='-DXBYAK_DONT_USE_MAP_JIT' -DUSE_AVX=OFF -DUSE_LIBXSMM=OFF .. -- Start configuring project dgl -- Disabling LIBXSMM on arm64. CMake Warning at CMakeLists.txt:197 (message): EPOLL is not available on this platform...

CMake Error at CMakeLists.txt:266 (add_subdirectory): The source directory /Users/haimonti/Research/GraphNeuralNetworks/Code/dgl/third_party/dmlc-core does not contain a CMakeLists.txt file.

CMake Error at CMakeLists.txt:273 (include): include could not find requested file: /Users/haimonti/Research/GraphNeuralNetworks/Code/dgl/third_party/METIS/GKlib/GKlibSystem.cmake

CMake Error at CMakeLists.txt:276 (add_subdirectory): add_subdirectory given source "third_party/METIS/libmetis/" which is not an existing directory.

Any suggestions? I did install Cython as suggested above, but that did not resolve the problem either.

Maninder416 commented 1 year ago

Using kafka stream with mac m2 pro I am getting this error:

2023-10-06 16:09:36.541 INFO 69179 --- [-StreamThread-1] c.o.l.k.s.ProductCategoryCodeOutput11 : product category info key: 11: value: BpaUlfProductCodes(id=11, product_cd=CAC, product_category_cd=CC) 2023-10-06 16:09:36.708 ERROR 69179 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [6830606d-cea5-4833-9018-e1d7522f4fd4-728c09bb-339a-4a06-8c74-b883960ac800] Encountered the following exception during processing and Kafka Streams opted to SHUTDOWN_CLIENT. The streams client is going to shut down now.

java.lang.UnsatisfiedLinkError: /private/var/folders/hj/rlzyd74168qf7184qly8zjjw0000gn/T/librocksdbjni12962212990828040106.jnilib: dlopen(/private/var/folders/hj/rlzyd74168qf7184qly8zjjw0000gn/T/librocksdbjni12962212990828040106.jnilib, 0x0001): tried: '/private/var/folders/hj/rlzyd74168qf7184qly8zjjw0000gn/T/librocksdbjni12962212990828040106.jnilib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/hj/rlzyd74168qf7184qly8zjjw0000gn/T/librocksdbjni12962212990828040106.jnilib' (no such file), '/private/var/folders/hj/rlzyd74168qf7184qly8zjjw0000gn/T/librocksdbjni12962212990828040106.jnilib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')) at java.base/jdk.internal.loader.NativeLibraries.load(Native Method) ~[na:na] at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388) ~[na:na] at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232) ~[na:na] at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174) ~[na:na] at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2394) ~[na:na] at java.base/java.lang.Runtime.load0(Runtime.java:755) ~[na:na] at java.base/java.lang.System.load(System.java:1953) ~[na:na] at org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:79) ~[rocksdbjni-6.22.1.1.jar:na] at org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:57) ~[rocksdbjni-6.22.1.1.jar:na] at org.rocksdb.RocksDB.loadLibrary(RocksDB.java:69) ~[rocksdbjni-6.22.1.1.jar:na] at org.rocksdb.RocksDB.(RocksDB.java:38) ~[rocksdbjni-6.22.1.1.jar:na] at org.rocksdb.DBOptions.(DBOptions.java:22) ~[rocksdbjni-6.22.1.1.jar:na] at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:126) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.KeyValueSegment.openDB(KeyValueSegment.java:56) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.KeyValueSegments.getOrCreateSegment(KeyValueSegments.java:51) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.KeyValueSegments.getOrCreateSegment(KeyValueSegments.java:26) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.AbstractSegments.getOrCreateSegmentIfLive(AbstractSegments.java:85) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.AbstractRocksDBSegmentedBytesStore.put(AbstractRocksDBSegmentedBytesStore.java:214) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.RocksDBWindowStore.put(RocksDBWindowStore.java:47) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.RocksDBWindowStore.put(RocksDBWindowStore.java:25) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:135) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:36) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.MeteredWindowStore.lambda$put$2(MeteredWindowStore.java:187) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:809) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.state.internals.MeteredWindowStore.put(MeteredWindowStore.java:186) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.AbstractReadWriteDecorator$WindowStoreReadWriteDecorator.put(AbstractReadWriteDecorator.java:168) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.kstream.internals.KStreamJoinWindow$KStreamJoinWindowProcessor.process(KStreamJoinWindow.java:57) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:146) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forwardInternal(ProcessorContextImpl.java:253) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:232) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:191) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:84) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.StreamTask.lambda$process$1(StreamTask.java:731) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:809) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:731) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:1296) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:784) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:604) ~[kafka-streams-3.1.2.jar:na] at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:576) ~[kafka-streams-3.1.2.jar:na]

dmlc / dgl