Closed zhanghang1989 closed 7 years ago
You either need to install a newer toolkit (@ngimel does the newest support apple clang 9.0?), or downgrade your command line tools. It's not a PyTorch problem
Thanks @apaszke Solution:
Command Line Tool for 8.3.2
from https://developer.apple.com/download/more/clang --version
:
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin17.0.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Then install PyTorch again.
With the same setup as you I get error: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.9" but "10.13" during configure
near the end. Anyone else experienced this?
Hi @zhanghang1989, Can I still use my current Xcode 9.0. Or should I get Xcode 8.3 before downloading the Command Line Tool for 8.3? Thanks.
@mtv3673crawlab you only need to download the CLT
I got that, thanks.
@ulfaslak did you manage to solve this? I'm stuck at this error.
I have High Sierra OS with Xcode 9.2 and cuda 9, and the following worked:
Command Line Tool for 8.2 (Dec 13, 2016)
sudo xcode-select --switch /Library/Developer/CommandLineTools
I did try to install with Command Line Tool for 8.3.2
but it failed.
Important TIP
Do MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py clean
before trying to install again
Why was this ticket closed? It's a serious issue leaving an entire platform (OS X) out of the support map and the solution above was not fully validated. I have tried all plausible downgrades of the CLI and none work; this "fix" was probably not a fix at all, e.g. not what made the code eventually build for the test case. This is a seriously overlooked nvidia/xcode compatibility issue I've seen in other related repos, claiming the "fix" to be in another CLI version which doesn't actually work either. Specifically with the 8.2 CLI (actually from Dec 12, not 13):
clang --version Apple LLVM version 8.0.0 (clang-800.0.42.1) Target: x86_64-apple-darwin16.7.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Errors: nvcc fatal : The version ('80000') of the host compiler ('Apple clang') is not supported ... CMake Error at ATen_generated_THCStorageCopy.cu.o.cmake:207 (message): ... etc.
Additionally, why the 10.9 target? Where was that inherited from? Should we take that as a hint as to how long ago has this code been actually tested for OS X?
the ticket was closed because there's literally nothing we can actually do about this problem. CUDA support on OSX is a nightmare, particular versions of cli tools are needed, coupled with particular versions of CUDA. One option is to upgrade your CUDA to 9.0 / 9.1 and see if that helps.
Should we take that as a hint as to how long ago has this code been actually tested for OS X?
We test on OS X, see https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-macos-10.13-py3-build-test/
Arguably we should detect OS X and set 10.9 in our scripts... OS X is weird and gives you a broken C++ toolchain if you don't set this correctly.
10.9 just means use stdlib that can run atleast on 10.9. the default is 10.6, which doesn't have c++11
10.9 just means use stdlib that can run atleast on 10.9. the default is 10.6, which doesn't have c++11 -> ambiguous, don't you think?
We test on OS X, see https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-macos-10.13-py3-build-test/ -> Why aren't the specs for that successful built released? There's a lot of people in lots of forums experiencing problems with this. Please give us a ground truth build test to start from. I take that anaconda was not involved, btw?
@datascienceteam01 Referenced an issue where I had similar problems. (https://github.com/pytorch/pytorch/issues/5244)
I had issues getting macOS to build PyTorch, it seemed to fail randomly, and only by trying repeatedly (with the same options, literally just re-running the same command over and over again was I able to successfully build).
It would be nice if a binary distribution with CUDA support was available, so users don't have to build it themselves. I'm not blaming PyTorch in any way, I think my problem may have been with NVCC. But, it seems like most people are unable to use PyTorch on macOS. Unfortunately, I'm afraid to help debug, because I don't want to lose my working build. It took me a couple days of fiddling to get it setup.
https://github.com/pytorch/pytorch/issues/5244 walks through everything I tried, maybe it will help @datascienceteam01. I use fish shell so the commands I reference will be slightly different if you're using bash.
@rudedogg I'm afraid that rolled binary distributions might not solve the issue. Pytorch is successfully becoming a dependency to critical cutting edge libraries in data science. NVIDIA has had issues with Apple for a while now. There's a lack of priority in CUDA support for Mac computers. No intent to polemize, it would be just good to see the pytorch team bring this issue to the right people at NVIDIA, a company that is actually affiliated with the project and is not, as suggested before, some entire third party. This is a seriously delaying issue to any scientist on a Mac.
@datascienceteam01 I am happy to flag it to the nvidia engineers. cc: @ngimel @csarofeen would you mind reporting this to the cuda team :)
I was having the same issue when trying to set up CUDA, your solution solved the problem for me. However now I get the following error, did anyone come across the same problem? I don't seem to find a way to fix it
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
For posterity I'd like to add my successful build specs:
I followed this tutorial but used the following versions. I also MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py clean
after every failed build.
macOS 10.13.4
XCode 9.3
CUDA 9.1 Toolkit
cuDNN 6.0
CLT for 8.3.2 (18 April 2017) + sudo xcode-select --switch /Library/Developer/CommandLineTools
Sadly the console of an app I was testing told me
Found GPU0 GeForce GT 750M which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
Built successfully with:
Python 3.6.4 macOS 10.13.4 CUDA 9.2 cuDNN 7.1 CLT for 8.3.2 latest (as of time of posting) versions of the build dependencies.
I used MACOSX_DEPLOYMENT_TARGET=10.13 CC=clang CXX=clang++ python setup.py install
to build successfully. This worked the first time I tried it.
Previously, I've had several unsuccessful builds using
MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
These raised
error: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.9" but "10.13" during configure
at the end of the build process.
I used MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py clean
after every failed build.
side note – I didn't use Anaconda; I used venv (installing the dependencies with pip), and pointed to the virtual environment root directory (e.g. .virtualenvs/DL/) instead of the anaconda root.
Please keep this ticket open until the proposed solution is properly tested. macOS CUDA support has not improved
I have very similar configs to what @yordun specified, but the following step keeps failing:
[ 97%] Building CXX object caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/ATen/cudnn/Types.cpp.o warning: unknown warning option '-Wno-invalid-partial-specialization' [-Wunknown-warning-option] 1 warning generated. 1 warning generated. 1 warning generated. 1 warning generated. 1 warning generated. 1 warning generated. 1 warning generated. 1 warning generated. [ 97%] Linking CXX shared library ../lib/libcaffe2_gpu.dylib ld: can't map file, errno=22 file '/Library/Frameworks/cuda.framework' for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: [lib/libcaffe2_gpu.1.dylib] Error 1 make[1]: [caffe2/CMakeFiles/caffe2_gpu.dir/all] Error 2 make: *** [all] Error 2 Failed to run 'bash tools/build_pytorch_libs.sh --with-cuda --with-nnpack caffe2 nanopb libshm THD'
I've confirmed that the directory /Library/Frameworks/cuda.framework exists, but am not sure why this is happening. Would appreciate any advice.
wow - same issue here
when build starts - it prints:
-- CUDA library : /Library/Frameworks/cuda.framework
-- cudart library : /Developer/NVIDIA/CUDA-8.0/lib/libcudart_static.a;-Wl,-rpath,/usr/local/cuda/lib
this should be
-- CUDA library : /Developer/NVIDIA/CUDA-8.0/lib/libcuda.dylib
-- cudart library : /Developer/NVIDIA/CUDA-8.0/lib/libcudart_static.a;-Wl,-rpath,/usr/local/cuda/lib
I have the same issue
-- CUDA root directory : /usr/local/cuda -- CUDA library : /Library/Frameworks/cuda.framework -- cudart library : /usr/local/cuda/lib/libcudart_static.a
ld: can't map file, errno=22 file '/Library/Frameworks/cuda.framework' for architecture x86_64 clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: [lib/libcaffe2_gpu.1.dylib] Error 1 make[1]: [caffe2/CMakeFiles/caffe2_gpu.dir/all] Error 2 make: *** [all] Error 2
MacOS 10.13.3 + MacOS 10.13.4 Cuda 9.1 + Cuda 9.2 CLT for 8.3.2
Trying to build from scratch on Mac OS X 10.13.4 (17E202)
clang --version
clang version 6.0.0 (tags/RELEASE_600/final)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /usr/local/miniconda3/bin
-> nvcc fatal : The version ('6.0') of the host compiler ('clang') is not supported
So what version am I supposed to get if not the latest that is shipped with Miniconda? Furthermore it looks like the CC/CXX environment variables are totally ignored. Pointed them to the Apple's clang
which has a version of its own but the setup still tries to access clang
on the Miniconda path.
Well, reading the above - it did work for others at some time - I checked some revisions and in fact 200fb22b22c5f1c5345d99743ef43764b9a8323c worked fine for me.
With an activated conda environment :
export CMAKE_PREFIX_PATH=[anaconda root directory]
conda install numpy pyyaml mkl mkl-include setuptools cmake cffi typing
git reset --hard 200fb22b22c5f1c5345d99743ef43764b9a8323c
and
git submodule update --init
finally
MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
Mac OS X 10.13.4 (17E202) Cuda 9.2 CLT for 8.3.2
Some updates in caffe2 seem to mess things up with the Cuda library path in MacOs.
I understand they're reworking a lot of stuff before going 1.0
. To be safe, one can probably use 0.4.0
, which should be stable
I had an issue with
The version ('6.0') of the host compiler ('clang') is not supported
It's because of installed LLVM
and we somehow need to know cmake that we want to use 'clang' and not llvm - in that case need to execute command:
MACOSX_DEPLOYMENT_TARGET=10.13 DCMAKE_C_COMPILER=clang DCMAKE_CXX_COMPILER=clang++ python setup.py install
Also sometimes if you still have such problem - it's better to download XCode 8.3.3 from https://developer.apple.com/download/more/ and temporarily replace XCode 9.X for pytorch build. At least it also helped me :)
So each version of CUDA requires a specific version of Xcode. You'll have to check which version of CUDA you have installed and which version of Xcode it requires, then download and select the command line tools of that version.
Check your version of CUDA:
ls /Developer/NVIDIA
In my case that was CUDA 9.2. Check here for the documentation of your version and go to the Installation Guide for Mac OS X, there under System Requirements you'll find a table with the supported Xcode version(s). For CUDA 9.1 and 9.2 that is Xcode 9.2.
Check your version of Xcode:
xcodebuild -version
If it matches the one required by CUDA, you're good to go! Otherwise go to the Apple Developer Downloads Page and download the Command Line Tools matching the Xcode version required by CUDA and install them.
Activate the command line tools:
sudo xcode-select --switch /Library/Developer/CommandLineTools
Build and install Pytorch:
MACOSX_DEPLOYMENT_TARGET=10.13 CC=clang CXX=clang++ python setup.py install
After the installation completed successfully, you might to want to reset the command line tools to the latest version of Xcode:
sudo xcode-select --reset
Hello I have an error with the command MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install I'm trying to install Pytorch for Mac I’m following this tutorial : https://www.cs.rochester.edu/u/kautz/Installing-Pytorch-Cuda-on-Macbook.html
I’m using macOS 10.13.4 I have installed:
I installed pytorch with the following command:
conda install pytorch=0.4.1 -c pytorch
I did that because conda install pytorch=0.4.1 cuda90 -c pytorch did not work. I think the whl file is not longer available on https://pytorch.org/get-started/previous-versions/
So after I did that, I ran the command line MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
Now I get this :
running install running build_deps
-Wall -Wno-unused -Wno-attributes -Wno-unused-result -ffp-contract=off -fno-math-errno -fno-trapping-math
to compile libsleef
-- Building shared libs : OFF
-- MPFR : /usr/local/lib/libmpfr.dylib
-- MPFR header file in /usr/local/include
-- GMP : /usr/local/lib/libgmp.dylib
-- RUNNING_ON_TRAVIS : 0
-- COMPILER_SUPPORTS_OPENMP :
CMake Warning at CMakeLists.txt:346 (message):
Generated cmake files are only fully tested if one builds with system glog,
gflags, and protobuf. Other settings may generate files that are not well
tested.-- -- **** Summary **** -- General: -- CMake version : 3.12.2 -- CMake command : /anaconda3/envs/ptc/bin/cmake -- Git version : -- System : Darwin -- C++ compiler : /Users/demanouwamo/Downloads/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -- C++ compiler version : 8.1.0.8020042 -- BLAS : MKL -- CXX flags : -msse4.1 -msse4.2 --std=c++11 -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations -DONNX_NAMESPACE=onnx_torch -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-aliasing -Wno-error=deprecated-declarations -- Build type : Release -- Compile definitions : USE_GCC_ATOMICS=1;TH_BLAS_MKL;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1 -- CMAKE_PREFIX_PATH : /anaconda3/envs/ptc/lib/python3.6/site-packages -- CMAKE_INSTALL_PREFIX : /Users/demanouwamo/Downloads/pytorch/torch/lib/tmp_install -- -- BUILD_CAFFE2 : 0 -- BUILD_ATEN : ON -- BUILD_BINARY : OFF -- BUILD_CUSTOM_PROTOBUF : ON -- Link local protobuf : ON -- BUILD_PYTHON : 0 -- BUILD_SHARED_LIBS : ON -- BUILD_TEST : OFF -- USE_ASAN : OFF -- USE_ATEN : OFF -- USE_CUDA : 1 -- CUDA static link : -- USE_CUDNN : ON -- CUDA version : 9.0 -- cuDNN version : 7.0.4 -- CUDA root directory : /usr/local/cuda -- CUDA library : /usr/local/cuda/lib/libcuda.dylib -- cudart library : /usr/local/cuda/lib/libcudart_static.a;-Wl,-rpath,/usr/local/cuda/lib -- cublas library : /usr/local/cuda/lib/libcublas.dylib;/usr/local/cuda/lib/libcublas_device.a -- cufft library : /usr/local/cuda/lib/libcufft.dylib -- curand library : /usr/local/cuda/lib/libcurand.dylib -- cuDNN library : /usr/local/cuda/lib/libcudnn.7.dylib -- nvrtc : /usr/local/cuda/lib/libnvrtc.dylib -- CUDA include path : /usr/local/cuda/include -- NVCC executable : /usr/local/cuda/bin/nvcc -- CUDA host compiler : /Users/demanouwamo/Downloads/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -- USE_TENSORRT : OFF -- USE_ROCM : OFF -- USE_EIGEN_FOR_BLAS : -- USE_FFMPEG : OFF -- USE_GFLAGS : OFF -- USE_GLOG : OFF -- USE_GLOO : OFF -- USE_LEVELDB : OFF -- USE_LITE_PROTO : OFF -- USE_LMDB : OFF -- USE_METAL : OFF -- USE_MKL : -- USE_MOBILE_OPENGL : OFF -- USE_MPI : OFF -- USE_NCCL : OFF -- USE_NERVANA_GPU : OFF -- USE_NNPACK : 1 -- USE_OBSERVERS : OFF -- USE_OPENCL : OFF -- USE_OPENCV : OFF -- USE_OPENMP : OFF -- USE_PROF : OFF -- USE_REDIS : OFF -- USE_ROCKSDB : OFF -- USE_ZMQ : OFF -- USE_DISTRIBUTED : OFF -- Public Dependencies : /anaconda3/envs/ptc/lib/libmkl_intel_lp64.dylib;/anaconda3/envs/ptc/lib/libmkl_intel_thread.dylib;/anaconda3/envs/ptc/lib/libmkl_core.dylib;/anaconda3/envs/ptc/lib/libiomp5.dylib;/usr/lib/libpthread.dylib;/usr/lib/libm.dylib -- Private Dependencies : nnpack;cpuinfo;onnxifi_loader -- Configuring done -- Generating done -- Build files have been written to: /Users/demanouwamo/Downloads/pytorch/build
nvcc fatal : Unsupported gpu architecture 'compute_70' CMake Error at caffe2_gpu_generated_THCBlas.cu.o.Release.cmake:219 (message): Error generating /Users/demanouwamo/Downloads/pytorch/build/caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/THC/./caffe2_gpu_generated_THCBlas.cu.o
CMake Error at caffe2_gpu_generated_THCReduceApplyUtils.cu.o.Release.cmake:219 (message): Error generating /Users/demanouwamo/Downloads/pytorch/build/caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/THC/./caffe2_gpu_generated_THCReduceApplyUtils.cu.o
make[2]: [caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/THC/caffe2_gpu_generated_THCSleep.cu.o] Error 1 make[2]: Waiting for unfinished jobs.... make[2]: [caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/THC/caffe2_gpu_generated_THCBlas.cu.o] Error 1 make[2]: [caffe2/CMakeFiles/caffe2_gpu.dir//aten/src/THC/caffe2_gpu_generated_THCReduceApplyUtils.cu.o] Error 1 nvcc fatal : Unsupported gpu architecture 'compute_70' CMake Error at caffe2_gpu_generated_THCStorage.cu.o.Release.cmake:219 (message): Error generating /Users/demanouwamo/Downloads/pytorch/build/caffe2/CMakeFiles/caffe2_gpu.dir//aten/src/THC/./caffe2_gpu_generated_THCStorage.cu.o
make[2]: [caffe2/CMakeFiles/caffe2_gpu.dir/__/aten/src/THC/caffe2_gpu_generated_THCStorage.cu.o] Error 1 make[1]: [caffe2/CMakeFiles/caffe2_gpu.dir/all] Error 2 make: *** [all] Error 2 Failed to run 'bash tools/build_pytorch_libs.sh --use-cuda --use-nnpack caffe2 nanopb libshm THD
Could you help me please.
Hello @soumith sorry for the duplicate. Do you have any idea on what going wrong with my post
This is still a problem. I know Apple doesn't ship with NVidia boards any more but users can still use the boards in expansion chasis and in older machines. NVidia should support modern OS build environment if they ever hope to be taken seriously on the platform that most modern developers work on. The latest 10.2 CUDA still fails and requires an older mac OS to even install. I was able to get the compilers in place but they refuse to run against the host compiler version. That's on NVidia to fix. This is an embarrassment for NVidia in my opinion. This reflects very poorly. You make a critical technology with CUDA. You should be supporting it on all graphics boards -- even AMD.
Hello,
For the macOS Big Sur (11.4), I tried to install CUDA with pytorch but I encountered the same issue:
nvcc fatal : The version ('13.0') of the host compiler ('Apple clang') is not supported
CMake Error at torch_cuda_generated_AveragePool2d.cu.o.Release.cmake:221 (message):
I tried previous threads but it won't work. I wonder is there a solution? Any help will be appreciated.
Best, KS
nvcc fatal : The version ('90000') of the host compiler ('Apple clang') is not supported