search

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.34k stars 7.46k forks source link

Dockerfile installation failure: No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' #1772

Closed yeeunsong closed 4 years ago

yeeunsong commented 4 years ago

I'm building a Dockerfile on my Windows 10 machine (has 2 GPUs). I'm using the standard Docker, and followed the instructions you put up at here.

https://github.com/facebookresearch/detectron2/tree/master/docker

I'm using a docker container with docker>=19.03 , so I followed the below command.

  1. what exact command I ran: 
    docker build --build-arg USER_ID=1000 -t detectron2:v0 .

    It didn't work out with me with this USER_ID=$UID command , so I used the argument USER_ID=1000

Using the USER_ID=$UID returned with this log,

Step 6/21 : RUN useradd -m --no-log-init --system  --uid ${USER_ID} appuser -g sudo
 ---> Running in 791ea2a3f2e9
useradd: invalid user ID '$UID'
  1. Error message:
    The error happened at this stage: 
    Step 19/21 : RUN pip install --user -e detectron2_repo
---> Running in 6019a28d7ce1

And I got the below error message. 

 No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

  1. full logs you observed:

    ERROR: Command errored out with exit status 1:
 command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/appuser/detectron2_repo/setup.py'"'"'; __file__='"'"'/home/appuser/detectron2_repo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
     cwd: /home/appuser/detectron2_repo/
Complete output (147 lines):
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running develop
running egg_info
creating detectron2.egg-info
writing detectron2.egg-info/PKG-INFO
writing dependency_links to detectron2.egg-info/dependency_links.txt
writing requirements to detectron2.egg-info/requires.txt
writing top-level names to detectron2.egg-info/top_level.txt
writing manifest file 'detectron2.egg-info/SOURCES.txt'
reading manifest file 'detectron2.egg-info/SOURCES.txt'
writing manifest file 'detectron2.egg-info/SOURCES.txt'
running build_ext
building 'detectron2._C' extension
creating /home/appuser/detectron2_repo/build
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlign
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/nms_rotated
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/box_iou_rotated
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlignRotated
creating /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/deformable
Emitting ninja build file /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/13] c++ -MMD -MF /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.o.d -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp: In function 'void detectron2::COCOeval::MatchDetectionsToGroundTruth(const std::vector<detectron2::COCOeval::InstanceAnnotation, std::allocator<detectron2::COCOeval::InstanceAnnotation> >&, const std::vector<long unsigned int>&, const std::vector<detectron2::COCOeval::InstanceAnnotation, std::allocator<detectron2::COCOeval::InstanceAnnotation> >&, const std::vector<long unsigned int>&, const std::vector<bool>&, const std::vector<std::vector<double>, std::allocator<std::vector<double> > >&, const std::vector<double>&, const std::array<double, 2>&, detectron2::COCOeval::ImageEvaluation*)':
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:82:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (auto g = 0; g < num_ground_truth; ++g) {
                    ~~^~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:86:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (auto t = 0; t < num_iou_thresholds; ++t) {
                    ~~^~~~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:87:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (auto d = 0; d < num_detections; ++d) {
                      ~~^~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:91:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       for (auto g = 0; g < num_ground_truth; ++g) {
                        ~~^~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:135:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (auto d = 0; d < detection_sorted_indices.size(); ++d) {
                    ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp: In function 'std::vector<detectron2::COCOeval::ImageEvaluation> detectron2::COCOeval::EvaluateImages(const std::vector<std::array<double, 2> >&, int, const std::vector<double>&, detectron2::COCOeval::ImageCategoryInstances<std::vector<double> >&, detectron2::COCOeval::ImageCategoryInstances<detectron2::COCOeval::InstanceAnnotation>&, detectron2::COCOeval::ImageCategoryInstances<detectron2::COCOeval::InstanceAnnotation>&)':
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:162:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (auto i = 0; i < num_images; ++i) {
                    ~~^~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:163:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (auto c = 0; c < num_categories; ++c) {
                      ~~^~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:171:43: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       if (detection_sorted_indices.size() > max_detections) {
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:175:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       for (auto a = 0; a < area_ranges.size(); ++a) {
                        ~~^~~~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp: In function 'int64_t detectron2::COCOeval::BuildSortedDetectionList(const std::vector<detectron2::COCOeval::ImageEvaluation>&, int64_t, int64_t, int64_t, std::vector<long unsigned int>*, std::vector<double>*, std::vector<long unsigned int>*, std::vector<long unsigned int>*)':
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:246:12: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
          d < evaluation.detection_scores.size() && d < max_detections;
          ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp: In function 'void detectron2::COCOeval::ComputePrecisionRecallCurve(int64_t, int64_t, int64_t, const std::vector<double>&, int64_t, int64_t, int64_t, const std::vector<detectron2::COCOeval::ImageEvaluation>&, const std::vector<long unsigned int>&, const std::vector<double>&, const std::vector<long unsigned int>&, const std::vector<long unsigned int>&, std::vector<double>*, std::vector<double>*, std::vector<double>*, std::vector<double>*, std::vector<double>*)':
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:352:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (auto r = 0; r < recall_thresholds.size(); ++r) {
                    ~~^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:361:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (precisions_index < precisions->size()) {
         ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp: In instantiation of 'std::vector<T> detectron2::COCOeval::list_to_vec(const pybind11::list&) [with T = double]':
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:375:49:   required from here
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:204:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < py::len(l); ++i) {
                   ~~^~~~~~~~~~~~
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp: In instantiation of 'std::vector<T> detectron2::COCOeval::list_to_vec(const pybind11::list&) [with T = long int]':
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:377:50:   required from here
/home/appuser/detectron2_repo/detectron2/layers/csrc/cocoeval/cocoeval.cpp:204:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
[2/13] c++ -MMD -MF /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlign/ROIAlign_cpu.o.d -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlign/ROIAlign_cpu.cpp -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlign/ROIAlign_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
[3/13] c++ -MMD -MF /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/nms_rotated/nms_rotated_cpu.o.d -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/nms_rotated/nms_rotated_cpu.cpp -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/nms_rotated/nms_rotated_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
[4/13] c++ -MMD -MF /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_cpu.o.d -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_cpu.cpp -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
[5/13] c++ -MMD -MF /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlignRotated/ROIAlignRotated_cpu.o.d -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlignRotated/ROIAlignRotated_cpu.cpp -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlignRotated/ROIAlignRotated_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
[6/13] c++ -MMD -MF /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/vision.o.d -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/vision.cpp -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/vision.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/vision.o
c++ -MMD -MF /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/vision.o.d -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/vision.cpp -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/vision.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
[7/13] /usr/local/cuda/bin/nvcc -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -O3 -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_53,code=sm_53 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_37,code=sm_37 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_52,code=sm_52 -gencode=arch=compute_35,code=compute_35 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_50,code=sm_50 -std=c++14
[8/13] /usr/local/cuda/bin/nvcc -DWITH_CUDA -I/home/appuser/detectron2_repo/detectron2/layers/csrc -I/home/appuser/.local/lib/python3.6/site-packages/torch/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/TH -I/home/appuser/.local/lib/python3.6/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c -c /home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlign/ROIAlign_cuda.cu -o /home/appuser/detectron2_repo/build/temp.linux-x86_64-3.6/home/appuser/detectron2_repo/detectron2/layers/csrc/ROIAlign/ROIAlign_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -O3 -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_53,code=sm_53 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_37,code=sm_37 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_52,code=sm_52 -gencode=arch=compute_35,code=compute_35 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_50,code=sm_50 -std=c++14
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/appuser/.local/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1400, in _run_ninja_build
    check=True)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/appuser/detectron2_repo/setup.py", line 202, in <module>
    cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension},
  File "/home/appuser/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 165, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/appuser/.local/lib/python3.6/site-packages/setuptools/command/develop.py", line 38, in run
    self.install_for_development()
  File "/home/appuser/.local/lib/python3.6/site-packages/setuptools/command/develop.py", line 140, in install_for_development
    self.run_command('build_ext')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/appuser/.local/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 87, in run
    _build_ext.run(self)
  File "/home/appuser/.local/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/appuser/.local/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 580, in build_extensions
    build_ext.build_extensions(self)
  File "/home/appuser/.local/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/appuser/.local/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/home/appuser/.local/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 423, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/home/appuser/.local/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects
    error_prefix='Error compiling objects for extension')
  File "/home/appuser/.local/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1413, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error compiling objects for extension
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/appuser/detectron2_repo/setup.py'"'"'; __file__='"'"'/home/appuser/detectron2_repo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.
The command '/bin/sh -c pip install --user -e detectron2_repo' returned a non-zero code: 1

Expected behavior:

I expected for the docker building to well-end, but it keeps producing the error below.

 No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

Environment:

D:\_Intern\2_Detec_Docker\detectron2\detectron2\utils>python collect_env.py
----------------------  -------------------------------------------------------------------------------------
sys.platform            win32
Python                  3.7.6 (default, Jan  8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)]
numpy                   1.18.1
detectron2              failed to import
detectron2._C           failed to import
DETECTRON2_ENV_MODULE   <not set>
PyTorch                 1.5.1 @C:\Users\modman\anaconda3\lib\site-packages\torch
PyTorch debug build     False
GPU available           True
GPU 0                   GeForce GTX TITAN X
CUDA_HOME               C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2
Pillow                  7.0.0
torchvision             0.6.1 @C:\Users\modman\anaconda3\lib\site-packages\torchvision
torchvision arch flags  C:\Users\modman\anaconda3\lib\site-packages\torchvision\_C.pyd; cannot find cuobjdump
fvcore                  0.1.1
----------------------  -------------------------------------------------------------------------------------
PyTorch built with:
  - C++ Version: 199711
  - MSVC 191627041
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191125 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
  - OpenMP 200203
  - CPU capability usage: AVX2
  - CUDA Runtime 10.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  - CuDNN 7.6.4
  - Magma 2.5.2
  - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR  /w /EHa /bigobj -openmp -DNDEBUG -DUSE_FBGEMM, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

Thank you so much!

ppwwyyxx commented 4 years ago

I'm not able to reproduce this on Linux although I would hope docker is a fully reproducible environment. And since the log says it's a compiler error:

c++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See file:///usr/share/doc/gcc-7/README.Bugs for instructions.

it's unlikely that we can do much about it. It might help to update to latest version of source code, or switch to a different version of the compiler in the docker container.