Open jplu opened 2 years ago
snnn
commented
2 years ago I think the argument doesn't work properly. So I created a PR to remove it, but it didn't get enough eye sights. https://github.com/microsoft/onnxruntime/pull/8754
And a PR to clarify the build instructions: https://github.com/microsoft/onnxruntime/pull/8710
I would suggest you build it with
--cmake_extra_defines CMAKE_OSX_ARCHITECTURES=arm64Let me know if the error still exists.
snnn
commented
2 years ago For your information, the macOS binaries in our github release package were built in a way like:
./build.sh --build --build_java --build_nodejs --use_coreml --cmake_extra_defines CMAKE_OSX_ARCHITECTURES=arm64 --parallel --build_shared_lib --config ReleaseI think you don't need to use "--build_java --build_nodejs"
And we haven't tried it for training. I know training x86_64 works. I haven't tried arm. I hope there won't be problems.
jplu
commented
2 years ago Thanks! Now with this command line:
./build.sh --build_dir build --use_coreml --cmake_extra_defines CMAKE_OSX_ARCHITECTURES=arm64 --skip_tests --enable_training --parallel --build_shared_lib --config ReleaseI get a different compilation error but still related to protobuf:
[ 9%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf-lite.dir/__/src/google/protobuf/wire_format_lite.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/strutil.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/time.cc.o
/Users/jplu/dev/onnxruntime/cmake/external/protobuf/src/google/protobuf/stubs/strutil.cc:1255:19: error: expected unqualified-id
} else if (std::isnan(value)) {
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:165:5: note: expanded from macro 'isnan'
( sizeof(x) == sizeof(float) ? __inline_isnanf((float)(x)) \
^
/Users/jplu/dev/onnxruntime/cmake/external/protobuf/src/google/protobuf/stubs/strutil.cc:1373:19: error: expected unqualified-id
} else if (std::isnan(value)) {
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:165:5: note: expanded from macro 'isnan'
( sizeof(x) == sizeof(float) ? __inline_isnanf((float)(x)) \
^
2 errors generated.
make[2]: *** [external/protobuf/cmake/CMakeFiles/libprotobuf-lite.dir/__/src/google/protobuf/stubs/strutil.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wire_format_lite.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/any.cc.o
180 warnings generated.
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/any.pb.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/api.pb.cc.o
[ 10%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/compiler/importer.cc.o
/Users/jplu/dev/onnxruntime/cmake/external/protobuf/src/google/protobuf/stubs/strutil.cc:1255:19: error: expected unqualified-id
} else if (std::isnan(value)) {
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:165:5: note: expanded from macro 'isnan'
( sizeof(x) == sizeof(float) ? __inline_isnanf((float)(x)) \
^
/Users/jplu/dev/onnxruntime/cmake/external/protobuf/src/google/protobuf/stubs/strutil.cc:1373:19: error: expected unqualified-id
} else if (std::isnan(value)) {
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:165:5: note: expanded from macro 'isnan'
( sizeof(x) == sizeof(float) ? __inline_isnanf((float)(x)) \
^
2 errors generated.
make[2]: *** [external/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/stubs/strutil.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [external/protobuf/cmake/CMakeFiles/libprotobuf-lite.dir/all] Error 2
make[1]: *** [external/protobuf/cmake/CMakeFiles/libprotobuf.dir/all] Error 2
make: *** [all] Error 2
Traceback (most recent call last):
File "/Users/jplu/dev/onnxruntime/tools/ci_build/build.py", line 2493, in <module>
sys.exit(main())
File "/Users/jplu/dev/onnxruntime/tools/ci_build/build.py", line 2408, in main
build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
File "/Users/jplu/dev/onnxruntime/tools/ci_build/build.py", line 1245, in build_targets
run_subprocess(cmd_args, env=env)
File "/Users/jplu/dev/onnxruntime/tools/ci_build/build.py", line 673, in run_subprocess
return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
File "/Users/jplu/dev/onnxruntime/tools/python/util/run.py", line 42, in run
completed_process = subprocess.run(
File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/opt/homebrew/bin/cmake', '--build', 'build/Release', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.I haven't seen such error before. It looks like the compiler you are using isn't compatible with the protobuf version we are using. When the build just started, cmake should print out compiler information. Would you please post it here? Is it Apple Clang?_
jplu
commented
2 years ago Here the summary given by CMake:
--
-- ******** Summary ********
-- CMake version : 3.22.2
-- CMake command : /opt/homebrew/Cellar/cmake/3.22.2/bin/cmake
-- System : Darwin
-- C++ compiler : /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- C++ compiler version : 13.1.6.13160021
-- CXX flags : -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -fstack-protector-strong -DCPUINFO_SUPPORTED -Wno-deprecated -Wnon-virtual-dtor
-- Build type : Release
-- Compile definitions : EIGEN_MPL2_ONLY;ENABLE_CPU_FP16_TRAINING_OPS;PLATFORM_POSIX;__STDC_FORMAT_MACROS
-- CMAKE_PREFIX_PATH :
-- CMAKE_INSTALL_PREFIX : /usr/local
-- CMAKE_MODULE_PATH : /Users/jplu/dev/onnxruntime/cmake/external
--
-- ONNX version : 1.11.0
-- ONNX NAMESPACE : onnx
-- ONNX_USE_LITE_PROTO : ON
-- USE_PROTOBUF_SHARED_LIBS : OFF
-- Protobuf_USE_STATIC_LIBS : ON
-- ONNX_DISABLE_EXCEPTIONS : OFF
-- ONNX_WERROR : OFF
-- ONNX_BUILD_TESTS : OFF
-- ONNX_BUILD_BENCHMARKS : OFF
-- ONNXIFI_DUMMY_BACKEND : OFF
-- ONNXIFI_ENABLE_EXT : OFF
--
-- Protobuf compiler :
-- Protobuf includes :
-- Protobuf libraries :
-- BUILD_ONNX_PYTHON : OFF
Use flatbuffers from submodule
-- Checking for module 'mpi-c'
-- No package 'mpi-c' found
-- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS)
-- Checking for module 'mpi-cxx'
-- No package 'mpi-cxx' found
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
-- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND)
CMake Warning at CMakeLists.txt:1843 (message):
MPI is not found. Please define onnxruntime_MPI_HOME to specify the path
of MPI. Otherwise, NCCL will be disabled.
-- Configuring done
-- Generating doneAnd the compiler version that CMake uses:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ --version
Apple clang version 13.1.6 (clang-1316.0.21.2.3)
Target: arm64-apple-darwin21.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/binAnd for your information, this is what we have now:
-- CMake command : /usr/local/Cellar/cmake/3.23.1/bin/cmake
-- System : Darwin
-- C++ compiler : /Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- C++ compiler version : 12.0.0.12000032
-- CXX flags : -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -fstack-protector-strong -DCPUINFO_SUPPORTED -Wno-deprecated -Wnon-virtual-dtor
-- Build type : Release
-- Compile definitions : EIGEN_MPL2_ONLY;PLATFORM_POSIX;__STDC_FORMAT_MACROS
-- CMAKE_PREFIX_PATH :
-- CMAKE_INSTALL_PREFIX : /usr/local
-- CMAKE_MODULE_PATH : /Users/runner/work/1/s/cmake/external
--
-- ONNX version : 1.11.0
-- ONNX NAMESPACE : onnx
-- ONNX_USE_LITE_PROTO : ON
-- USE_PROTOBUF_SHARED_LIBS : OFF
-- Protobuf_USE_STATIC_LIBS : ON
-- ONNX_DISABLE_EXCEPTIONS : OFF
-- ONNX_WERROR : OFF
-- ONNX_BUILD_TESTS : OFF
-- ONNX_BUILD_BENCHMARKS : OFF
-- ONNXIFI_DUMMY_BACKEND : OFF
-- ONNXIFI_ENABLE_EXT : OFF
--
-- Protobuf compiler :
-- Protobuf includes :
-- Protobuf libraries :
-- BUILD_ONNX_PYTHON : OFFYours is newer. I will try to see if I can get it one, otherwise it's difficult for me to work on.
jplu
commented
2 years ago My version of Xcode is 13.3.1 and I'm on MacOS Monterey 12.3.1 if it may help you.
jplu
commented
2 years ago @snnn Any update on your side on testing the compilation on a newer environment?
Describe the bug Issue to compile ONNX Runtime with training enabled.
Urgency Would be perfect to get an answer before the end of the first week of May, means at most the 6th of May. As I have to propose a POC before that date.
System information
To Reproduce
Expected behavior Properly compile ONNX Runtime
Additional context The compilation error is:
I suspect to do not provide the proper parameters, but there is a lack of documentation on this part, the only thing I found is this page.
This bring me to another question, is training and fine tuning models from C++ with ONNX Runtime possible on MacOS M1 at least? If not it is useless for me to go deeper in this.
Thanks a lot in advance for any light you could provide on this issue.