Open yaojunr opened 4 years ago
This doesn't look like an error with the Apex source code, this looks like an error with the Cuda runtime source somehow. It could be a mismatch between Cuda and GCC versions. Can you try compiling Pytorch's builtin extensions?
$ cd pytorch_repo_dir/test/cpp_extensions
$ python setup.py install
check /usr/local/cuda/include/host_config.h and other headers which you may changed. I accidentally changed the file wrongly according to the tutorial.
The installation command is
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
The full log is as follows: /home/zxm/anaconda3/lib/python3.7/site-packages/pip/_internal/commands/install.py:283: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options. cmdoptions.check_install_build_global(options) Created temporary directory: /tmp/pip-ephem-wheel-cache-p4vtlsv6 Created temporary directory: /tmp/pip-req-tracker-zuuadv7g Created requirements tracker '/tmp/pip-req-tracker-zuuadv7g' Created temporary directory: /tmp/pip-install-yw4by8oy Processing /home/zxm/apex Created temporary directory: /tmp/pip-req-build-qok5db0u Added file:///home/zxm/apex to build tracker '/tmp/pip-req-tracker-zuuadv7g' Running setup.py (path:/tmp/pip-req-build-qok5db0u/setup.py) egg_info for package from file:///home/zxm/apex Running command python setup.py egg_info torch.version = 1.3.0+cu92 running egg_info creating /tmp/pip-req-build-qok5db0u/pip-egg-info/apex.egg-info writing /tmp/pip-req-build-qok5db0u/pip-egg-info/apex.egg-info/PKG-INFO writing dependency_links to /tmp/pip-req-build-qok5db0u/pip-egg-info/apex.egg-info/dependency_links.txt writing top-level names to /tmp/pip-req-build-qok5db0u/pip-egg-info/apex.egg-info/toplevel.txt writing manifest file '/tmp/pip-req-build-qok5db0u/pip-egg-info/apex.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-req-build-qok5db0u/pip-egg-info/apex.egg-info/SOURCES.txt' writing manifest file '/tmp/pip-req-build-qok5db0u/pip-egg-info/apex.egg-info/SOURCES.txt' /tmp/pip-req-build-qok5db0u/setup.py:43: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies! warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!") Source in /tmp/pip-req-build-qok5db0u has version 0.1, which satisfies requirement apex==0.1 from file:///home/zxm/apex Removed apex==0.1 from file:///home/zxm/apex from build tracker '/tmp/pip-req-tracker-zuuadv7g' Skipping wheel build for apex, due to binaries being disabled for it. Installing collected packages: apex Created temporary directory: /tmp/pip-record-moss9qo Running command /home/zxm/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-qok5db0u/setup.py'"'"'; file='"'"'/tmp/pip-req-build-qok5db0u/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' --cpp_ext --cudaext install --record /tmp/pip-record-moss9qo/install-record.txt --single-version-externally-managed --compile torch.version = 1.3.0+cu92 /tmp/pip-req-build-qok5db0u/setup.py:43: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies! warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!")Cleaning up... Removing source in /tmp/pip-req-build-qok5db0u Removed build tracker '/tmp/pip-req-tracker-zuuadv7g' ERROR: Command errored out with exit status 1: /home/zxm/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-qok5db0u/setup.py'"'"'; file='"'"'/tmp/pip-req-build-qok5db0u/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' --cpp_ext --cudaext install --record /tmp/pip-record-moss9qo/install-record.txt --single-version-externally-managed --compile Check the logs for full command output. Exception information: Traceback (most recent call last): File "/home/zxm/anaconda3/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 153, in _main status = self.run(options, args) File "/home/zxm/anaconda3/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 455, in run use_user_site=options.use_user_site, File "/home/zxm/anaconda3/lib/python3.7/site-packages/pip/_internal/req/init.py", line 62, in install_given_reqs **kwargs File "/home/zxm/anaconda3/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 888, in install cwd=self.unpacked_source_directory, File "/home/zxm/anaconda3/lib/python3.7/site-packages/pip/_internal/utils/subprocess.py", line 275, in runner spinner=spinner, File "/home/zxm/anaconda3/lib/python3.7/site-packages/pip/_internal/utils/subprocess.py", line 242, in call_subprocess raise InstallationError(exc_msg) pip._internal.exceptions.InstallationError: Command errored out with exit status 1: /home/zxm/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-qok5db0u/setup.py'"'"'; file='"'"'/tmp/pip-req-build-qok5db0u/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' --cpp_ext --cudaext install --record /tmp/pip-record-moss9qo/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.