Build pytorch MPS

[Bycob]

Requires https://github.com/jolibrain/deepdetect/pull/1524 to be merged

Requirements:

brew install lmdb boost eigen rapidjson libarchive curlpp utf8cpp gflags pkg-config opencv@4 libomp
# for pytorch
python3 -m pip install typing_extensions pyyaml

Build:

cmake .. -DUSE_BOOST_BACKTRACE=OFF -DUSE_HTTP_SERVER_OATPP=ON \
-DUSE_CAFFE=OFF -DUSE_TORCH=ON -DWARNING=OFF \
-DUSE_BOOST_BACKTRACE=OFF -DUSE_HDF5=OFF \
-DBUILD_TESTS=OFF -DBUILD_SPDLOG=ON -DBUILD_PROTOBUF=OFF

You may need to uninstall system spdlog so that it doesn't interfere with locally built version

If cmake does not find opencv you can add flags to help, for example: -DUSE_OPENCV_VERSION=4 -DOpenCV_DIR=/opt/homebrew/Cellar/opencv/4.7.0_7/lib/cmake/opencv4

===

Tested with versions:

protobuf=23.4
opencv=4.8.0_1
libomp=16.0.6
cmake=2.27.1
llvm=16.0.6

===

If necessary versions can be pinned on mac using

brew pin opencv

With this, OpenCV will no longer be updated when running brew update; brew upgrade

[Bycob]

TODO (possibly):

• [x] Find OpenMP correctly
• [x] Support GPU device with MPS (torch device = "mps")
• [x] Disable CUDA & CUDnn by default if on APPLE
• [x] Variant lib: clone internally?
• [ ] (Build with custom protobuf)
• [ ] (Fix warnings)
• [x] Assemble all commits

[beniz]

Almost working on M1 (first gen):

/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/System/Library/Frameworks/DiskArbitration.framework/Headers/DADisk.h:58:87: warning: declaration does not declare anything [-Wmissing-declarations]
extern const CFStringRef kDADiskDescriptionMediaPathKey;       /* ( CFString     ) */ API_AVAILABLE(macos(10.4));
                                                                                      ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/os/availability.h:91:196: note: expanded from macro 'API_AVAILABLE'
    #define API_AVAILABLE(...) __API_AVAILABLE_GET_MACRO(__VA_ARGS__,__API_AVAILABLE8, __API_AVAILABLE7, __API_AVAILABLE6, __API_AVAILABLE5, __API_AVAILABLE4, __API_AVAILABLE3, __API_AVAILABLE2, __API_AVAILABLE1, 0)(__VA_ARGS__)
                                                                                                                                                                                                   ^
10 warnings generated.
[100%] Linking CXX shared library ../lib/libtorch_cpu.dylib
Undefined symbols for architecture arm64:
  "google::protobuf::internal::ThreadSafeArena::thread_cache_", referenced from:
      google::protobuf::RepeatedField<long long>::GrowNoAnnotate(int, int) in libcaffe2_protos.a(torch.pb.cc.o)
      google::protobuf::RepeatedField<long long>::GrowNoAnnotate(int, int) in libcaffe2_protos.a(caffe2.pb.cc.o)
      google::protobuf::RepeatedField<float>::GrowNoAnnotate(int, int) in libcaffe2_protos.a(caffe2.pb.cc.o)
      google::protobuf::RepeatedField<int>::GrowNoAnnotate(int, int) in libcaffe2_protos.a(caffe2.pb.cc.o)
      google::protobuf::RepeatedField<double>::GrowNoAnnotate(int, int) in libcaffe2_protos.a(caffe2.pb.cc.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[5]: *** [lib/libtorch_cpu.dylib] Error 1
make[4]: *** [caffe2/CMakeFiles/torch_cpu.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [pytorch/src/pytorch-stamp/pytorch-build] Error 1
make[1]: *** [CMakeFiles/pytorch.dir/all] Error 2
make: *** [all] Error 2

Build says

-   TORCH_VERSION         : 2.0.0

so it may come from wrong torch version. Note: the 2.0.1 torch is in though, and pytorch src dir has 2.0.1 tag.

[Bycob]

Version of protobuf? On the M2 it seems to be 3.21.12

3.23.2

[mergify[bot]]

This pull request is now in conflict :(