search

pytorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration
https://pytorch.org
Other
82.54k stars 22.21k forks source link

/usr/include/c++/11.2.0/cstdlib:75:15: fatal error: stdlib.h: No such file or directory #73688

Closed zasdfgbnm closed 2 years ago

zasdfgbnm commented 2 years ago

🐛 Describe the bug

When compiling PyTorch, I am seeing the following error:

[5715/6576] Building CXX object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/cuda/detail/CUDAHooks.cpp.o
FAILED: caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/cuda/detail/CUDAHooks.cpp.o 
/usr/bin/c++ -DAT_PER_OPERATOR_HEADERS -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DIDEEP_USE_MKL -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTORCH_CUDA_BUILD_MAIN_LIB -DUSE_C10D_GLOO -DUSE_C10D_MPI -DUSE_C10D_NCCL -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_EXTERNAL_MZCRC -DUSE_NCCL -DUSE_RPC -DUSE_TENSORPIPE -D_FILE_OFFSET_BITS=64 -Dtorch_cuda_EXPORTS -isystem /usr/include -I/home/gaoxiang/pytorch-master/build/aten/src -I/home/gaoxiang/pytorch-master/aten/src -I/home/gaoxiang/pytorch-master/build -I/home/gaoxiang/pytorch-master -I/home/gaoxiang/pytorch-master/cmake/../third_party/benchmark/include -I/home/gaoxiang/pytorch-master/cmake/../third_party/cudnn_frontend/include -I/home/gaoxiang/pytorch-master/third_party/onnx -I/home/gaoxiang/pytorch-master/build/third_party/onnx -I/home/gaoxiang/pytorch-master/third_party/foxi -I/home/gaoxiang/pytorch-master/build/third_party/foxi -I/home/gaoxiang/pytorch-master/build/include -I/home/gaoxiang/pytorch-master/torch/csrc/distributed -I/home/gaoxiang/pytorch-master/aten/src/THC -I/home/gaoxiang/pytorch-master/aten/src/ATen/cuda -I/home/gaoxiang/pytorch-master/build/caffe2/aten/src -I/home/gaoxiang/pytorch-master/aten/../third_party/catch/single_include -I/home/gaoxiang/pytorch-master/aten/src/ATen/.. -I/home/gaoxiang/pytorch-master/c10/cuda/../.. -I/home/gaoxiang/pytorch-master/c10/.. -I/home/gaoxiang/pytorch-master/third_party/tensorpipe -I/home/gaoxiang/pytorch-master/build/third_party/tensorpipe -I/home/gaoxiang/pytorch-master/third_party/tensorpipe/third_party/libnop/include -I/home/gaoxiang/pytorch-master/torch/csrc/api -I/home/gaoxiang/pytorch-master/torch/csrc/api/include -isystem /home/gaoxiang/pytorch-master/build/third_party/gloo -isystem /home/gaoxiang/pytorch-master/cmake/../third_party/gloo -isystem /home/gaoxiang/pytorch-master/cmake/../third_party/googletest/googlemock/include -isystem /home/gaoxiang/pytorch-master/cmake/../third_party/googletest/googletest/include -isystem /home/gaoxiang/pytorch-master/third_party/protobuf/src -isystem /opt/intel/mkl/include -isystem /home/gaoxiang/pytorch-master/third_party/gemmlowp -isystem /home/gaoxiang/pytorch-master/third_party/neon2sse -isystem /home/gaoxiang/pytorch-master/third_party/XNNPACK/include -isystem /home/gaoxiang/pytorch-master/cmake/../third_party/eigen -isystem /opt/cuda/include -isystem /home/gaoxiang/pytorch-master/third_party/ideep/mkl-dnn/third_party/oneDNN/include -isystem /home/gaoxiang/pytorch-master/third_party/ideep/include -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -DHAVE_AVX512_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -g -fno-omit-frame-pointer -O0 -fPIC -DCAFFE2_USE_GLOO -DTH_HAVE_THREAD -Wall -Wextra -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-missing-field-initializers -Wno-write-strings -Wno-unknown-pragmas -Wno-type-limits -Wno-array-bounds -Wno-sign-compare -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-missing-braces -Wno-maybe-uninitialized -fvisibility=hidden -DTORCH_CUDA_BUILD_MAIN_LIB -std=gnu++14 -MD -MT caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/cuda/detail/CUDAHooks.cpp.o -MF caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/cuda/detail/CUDAHooks.cpp.o.d -o caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/cuda/detail/CUDAHooks.cpp.o -c /home/gaoxiang/pytorch-master/aten/src/ATen/cuda/detail/CUDAHooks.cpp
In file included from /usr/include/c++/11.2.0/ext/string_conversions.h:41,
                 from /usr/include/c++/11.2.0/bits/basic_string.h:6607,
                 from /usr/include/c++/11.2.0/string:55,
                 from /usr/include/c++/11.2.0/bits/locale_classes.h:40,
                 from /usr/include/c++/11.2.0/bits/ios_base.h:41,
                 from /usr/include/c++/11.2.0/ios:42,
                 from /usr/include/c++/11.2.0/istream:38,
                 from /usr/include/c++/11.2.0/sstream:38,
                 from /home/gaoxiang/pytorch-master/c10/macros/Macros.h:245,
                 from /home/gaoxiang/pytorch-master/c10/core/DeviceType.h:8,
                 from /home/gaoxiang/pytorch-master/c10/core/Device.h:3,
                 from /home/gaoxiang/pytorch-master/c10/core/Allocator.h:6,
                 from /home/gaoxiang/pytorch-master/aten/src/ATen/detail/CUDAHooksInterface.h:3,
                 from /home/gaoxiang/pytorch-master/aten/src/ATen/cuda/detail/CUDAHooks.h:1,
                 from /home/gaoxiang/pytorch-master/aten/src/ATen/cuda/detail/CUDAHooks.cpp:1:
/usr/include/c++/11.2.0/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
   75 | #include_next <stdlib.h>
      |               ^~~~~~~~~~
compilation terminated.

The failure seems to be caused by some recent changes in our build. If I remove the -isystem /usr/include from the compilation command, I will build this file successfully.

Versions

$ gcc --version
gcc (GCC) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

cc @malfet @seemethere

malfet commented 2 years ago

Can you run the entire collect_env script please? And does it succeed while building first 5000+ files?

zasdfgbnm commented 2 years ago

@malfet Yes, the first 5k+ files succeed. The complete environment is:

Collecting environment information...
PyTorch version: 1.12.0a0+git6396547
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Arch Linux (x86_64)
GCC version: (GCC) 11.2.0
Clang version: 13.0.1
CMake version: version 3.22.2
Libc version: glibc-2.35

Python version: 3.10.2 (main, Jan 15 2022, 19:56:27) [GCC 11.1.0] (64-bit runtime)
Python platform: Linux-5.16.11-arch1-2-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.6.55
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 2080 Ti

Nvidia driver version: 510.54
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.3.1
/usr/lib/libcudnn_adv_infer.so.8.3.1
/usr/lib/libcudnn_adv_train.so.8.3.1
/usr/lib/libcudnn_cnn_infer.so.8.3.1
/usr/lib/libcudnn_cnn_train.so.8.3.1
/usr/lib/libcudnn_ops_infer.so.8.3.1
/usr/lib/libcudnn_ops_train.so.8.3.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.12.0a0+gitec8d677
[pip3] torch-ucc==1.0.0
[pip3] torchani==2.2
[conda] Could not collect
zasdfgbnm commented 2 years ago

Related commit: https://github.com/pytorch/pytorch/commit/6302cdb9bc8bbd2f77d002db296f272ef882d2fc