ValueError: Unknown CUDA arch () or GPU not supported

jonpol01 commented 2 years ago

Traceback (most recent call last): File "setup.py", line 488, in 'clean': clean, File "/usr/local/lib/python3.6/dist-packages/setuptools/init.py", line 153, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.6/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install.py", line 74, in run self.do_egg_install() File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install.py", line 116, in do_egg_install self.run_command('bdist_egg') File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/local/lib/python3.6/dist-packages/setuptools/command/bdist_egg.py", line 164, in run cmd = self.call_command('install_lib', warn_dir=0) File "/usr/local/lib/python3.6/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command self.run_command(cmdname) File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install_lib.py", line 11, in run self.build() File "/usr/lib/python3.6/distutils/command/install_lib.py", line 109, in build self.run_command('build_ext') File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_ext.py", line 79, in run _build_ext.run(self) File "/usr/local/lib/python3.6/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "/usr/lib/python3.6/distutils/command/build_ext.py", line 339, in run self.build_extensions() File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 709, in build_extensions build_ext.build_extensions(self) File "/usr/local/lib/python3.6/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "/usr/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions self._build_extensions_serial() File "/usr/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial self.build_extension(ext) File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension _build_ext.build_extension(self, ext) File "/usr/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension depends=ext.depends) File "/usr/lib/python3.6/distutils/ccompiler.py", line 574, in compile self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts) File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 457, in unix_wrap_single_compile cflags = unix_cuda_flags(cflags) File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 424, in unix_cuda_flags cflags + _get_cuda_arch_flags(cflags)) File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 1575, in _get_cuda_arch_flags raise ValueError(f"Unknown CUDA arch ({arch}) or GPU not supported") ValueError: Unknown CUDA arch () or GPU not supported The command '/bin/sh -c git clone -b ${TORCHVISION_VERSION} https://github.com/pytorch/vision torchvision && cd torchvision && python3 setup.py install && cd ../ && rm -rf torchvision' returned a non-zero code: 1

Need help on this error.. tried the command ./scripts/docker_build_ml.sh pytorchthis on a fresh Xavier NX Jetpack 4.6 [L4T 32.6.1].

Things did already

extra. bootfromexternalstorage
clone this repo, default-runtime-nvidia. done

dusty-nv commented 2 years ago

Hi @jonpol01, sorry for the delay - can I get your entire build log so that I can see which CUDA compute arch's are being set?

kevin00730 commented 2 years ago

Same error on NVIDIA Tegra X2 Run ./scripts/docker_build_ml.sh all Below are entire build logs

root@tegra-ubuntu:/usr/sdcard/jetson-containers# ./scripts/docker_build_ml.sh all reading L4T version from /etc/nv_tegra_release L4T BSP Version: L4T R32.6.1 l4t-base image: nvcr.io/nvidia/l4t-base:r32.6.1 selecting OpenCV for L4T R32.6.1... OPENCV_URL=https://nvidia.box.com/shared/static/5v89u6g5rb62fpz4lh0rz531ajo2t5ef.gz OPENCV_DEB=OpenCV-4.5.0-aarch64.tar.gz Python3 version: 3.6 building PyTorch torch-1.9.0-cp36-cp36m-linux_aarch64.whl, torchvision v0.10.0, torchaudio v0.9.0, cuda arch 5.3;6.2;7.2; Building l4t-pytorch:r32.6.1-pth1.9-py3 container... Sending build context to Docker daemon 178.2kB Step 1/26 : ARG BASE_IMAGE=nvcr.io/nvidia/l4t-base:r32.4.4 Step 2/26 : FROM ${BASE_IMAGE} ---> 1735192d3d51 Step 3/26 : ENV DEBIAN_FRONTEND=noninteractive ---> Using cache ---> ca9a5a9c6c8f Step 4/26 : RUN apt-get update && apt-get install -y --no-install-recommends python3-pip python3-dev libopenblas-dev libopenmpi-dev openmpi-bin openmpi-common gfortran libomp-dev && rm -rf /var/lib/apt/lists/ && apt-get clean ---> Using cache ---> b496d92f9383 Step 5/26 : RUN pip3 install --no-cache-dir setuptools Cython wheel ---> Using cache ---> 47674f2963d0 Step 6/26 : RUN pip3 install --no-cache-dir --verbose numpy ---> Using cache ---> d651682d906f Step 7/26 : ARG PYTORCH_URL=https://nvidia.box.com/shared/static/lufbgr3xu2uha40cs9ryq1zn4kxsnogl.whl ---> Using cache ---> 781a2295e37c Step 8/26 : ARG PYTORCH_WHL=torch-1.2.0-cp36-cp36m-linux_aarch64.whl ---> Using cache ---> 9bf0ca72e1ef Step 9/26 : RUN wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate ${PYTORCH_URL} -O ${PYTORCH_WHL} && pip3 install --no-cache-dir --verbose ${PYTORCH_WHL} && rm ${PYTORCH_WHL} ---> Using cache ---> 5187b58092dc Step 10/26 : ARG TORCHVISION_VERSION=v0.4.0 ---> Using cache ---> a280157bebca Step 11/26 : ARG TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2;8.7" ---> Using cache ---> 10064ce27a0d Step 12/26 : RUN printenv && echo "torchvision version = $TORCHVISION_VERSION" && echo "TORCH_CUDA_ARCH_LIST = $TORCH_CUDA_ARCH_LIST" ---> Using cache ---> f5c07a5d26f5 Step 13/26 : RUN apt-get update && apt-get install -y --no-install-recommends git build-essential libjpeg-dev zlib1g-dev && rm -rf /var/lib/apt/lists/ && apt-get clean ---> Using cache ---> 73685779ae15 Step 14/26 : RUN git clone -b ${TORCHVISION_VERSION} https://github.com/pytorch/vision torchvision && cd torchvision && python3 setup.py install && cd ../ && rm -rf torchvision ---> Running in abd91d638457 Cloning into 'torchvision'... Note: checking out '300a8a40132c5d3ee7dc936034c6046fd30ca4c0'.

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

Building wheel torchvision-0.10.0a0+300a8a4 PNG found: False Running build on conda-build: False Running build on conda: False JPEG found: True Building torchvision with JPEG image support NVJPEG found: False FFmpeg found: False running install running bdist_egg running egg_info creating torchvision.egg-info writing torchvision.egg-info/PKG-INFO writing dependency_links to torchvision.egg-info/dependency_links.txt writing requirements to torchvision.egg-info/requires.txt writing top-level names to torchvision.egg-info/top_level.txt writing manifest file 'torchvision.egg-info/SOURCES.txt' reading manifest file 'torchvision.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' /usr/local/lib/python3.6/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. setuptools.SetuptoolsDeprecationWarning, /usr/local/lib/python3.6/dist-packages/setuptools/command/easy_install.py:159: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. EasyInstallDeprecationWarning, /usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py:370: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.')) warning: no previously-included files matching 'pycache' found under directory '' warning: no previously-included files matching '.py[co]' found under directory '*' adding license file 'LICENSE' writing manifest file 'torchvision.egg-info/SOURCES.txt' installing library code to build/bdist.linux-aarch64/egg running install_lib running build_py creating build creating build/lib.linux-aarch64-3.6 creating build/lib.linux-aarch64-3.6/torchvision copying torchvision/extension.py -> build/lib.linux-aarch64-3.6/torchvision copying torchvision/utils.py -> build/lib.linux-aarch64-3.6/torchvision copying torchvision/version.py -> build/lib.linux-aarch64-3.6/torchvision copying torchvision/init.py -> build/lib.linux-aarch64-3.6/torchvision creating build/lib.linux-aarch64-3.6/torchvision/io copying torchvision/io/_video_opt.py -> build/lib.linux-aarch64-3.6/torchvision/io copying torchvision/io/video.py -> build/lib.linux-aarch64-3.6/torchvision/io copying torchvision/io/image.py -> build/lib.linux-aarch64-3.6/torchvision/io copying torchvision/io/init.py -> build/lib.linux-aarch64-3.6/torchvision/io creating build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/transforms.py -> build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/_transforms_video.py -> build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/functional.py -> build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/functional_tensor.py -> build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/autoaugment.py -> build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/_functional_video.py -> build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/init.py -> build/lib.linux-aarch64-3.6/torchvision/transforms copying torchvision/transforms/functional_pil.py -> build/lib.linux-aarch64-3.6/torchvision/transforms creating build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/kinetics.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/kitti.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/sbd.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/omniglot.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/phototour.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/widerface.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/cityscapes.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/usps.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/lsun.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/_utils.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/sbu.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/utils.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/flickr.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/caltech.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/hmdb51.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/voc.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/vision.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/places365.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/celeba.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/coco.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/svhn.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/mnist.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/fakedata.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/semeion.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/stl10.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/imagenet.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/video_utils.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/cifar.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/folder.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/ucf101.py -> build/lib.linux-aarch64-3.6/torchvision/datasets copying torchvision/datasets/init.py -> build/lib.linux-aarch64-3.6/torchvision/datasets creating build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/densenet.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/googlenet.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/inception.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/_utils.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/utils.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/resnet.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/mobilenetv2.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/alexnet.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/squeezenet.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/shufflenetv2.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/mobilenet.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/mobilenetv3.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/mnasnet.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/init.py -> build/lib.linux-aarch64-3.6/torchvision/models copying torchvision/models/vgg.py -> build/lib.linux-aarch64-3.6/torchvision/models creating build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/ps_roi_align.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/_utils.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/poolers.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/ps_roi_pool.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/feature_pyramid_network.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/focal_loss.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/boxes.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/roi_align.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/deform_conv.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/_box_convert.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/_register_onnx_ops.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/roi_pool.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/init.py -> build/lib.linux-aarch64-3.6/torchvision/ops copying torchvision/ops/misc.py -> build/lib.linux-aarch64-3.6/torchvision/ops creating build/lib.linux-aarch64-3.6/torchvision/datasets/samplers copying torchvision/datasets/samplers/clip_sampler.py -> build/lib.linux-aarch64-3.6/torchvision/datasets/samplers copying torchvision/datasets/samplers/init.py -> build/lib.linux-aarch64-3.6/torchvision/datasets/samplers creating build/lib.linux-aarch64-3.6/torchvision/models/segmentation copying torchvision/models/segmentation/fcn.py -> build/lib.linux-aarch64-3.6/torchvision/models/segmentation copying torchvision/models/segmentation/_utils.py -> build/lib.linux-aarch64-3.6/torchvision/models/segmentation copying torchvision/models/segmentation/deeplabv3.py -> build/lib.linux-aarch64-3.6/torchvision/models/segmentation copying torchvision/models/segmentation/segmentation.py -> build/lib.linux-aarch64-3.6/torchvision/models/segmentation copying torchvision/models/segmentation/lraspp.py -> build/lib.linux-aarch64-3.6/torchvision/models/segmentation copying torchvision/models/segmentation/init.py -> build/lib.linux-aarch64-3.6/torchvision/models/segmentation creating build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/googlenet.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/inception.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/utils.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/resnet.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/mobilenetv2.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/shufflenetv2.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/mobilenet.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/mobilenetv3.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization copying torchvision/models/quantization/init.py -> build/lib.linux-aarch64-3.6/torchvision/models/quantization creating build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/ssdlite.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/anchor_utils.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/image_list.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/keypoint_rcnn.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/_utils.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/mask_rcnn.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/retinanet.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/backbone_utils.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/roi_heads.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/rpn.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/faster_rcnn.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/ssd.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/init.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/generalized_rcnn.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection copying torchvision/models/detection/transform.py -> build/lib.linux-aarch64-3.6/torchvision/models/detection creating build/lib.linux-aarch64-3.6/torchvision/models/video copying torchvision/models/video/resnet.py -> build/lib.linux-aarch64-3.6/torchvision/models/video copying torchvision/models/video/init.py -> build/lib.linux-aarch64-3.6/torchvision/models/video running build_ext building 'torchvision._C' extension creating build/temp.linux-aarch64-3.6 creating build/temp.linux-aarch64-3.6/torchvision creating build/temp.linux-aarch64-3.6/torchvision/torchvision creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autocast creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autograd creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cuda creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/quantized creating build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/quantized/cpu aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autocast/deform_conv2d_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autocast/deform_conv2d_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14 aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autocast/nms_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autocast/nms_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14 aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autocast/ps_roi_align_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autocast/ps_roi_align_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14 aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autocast/ps_roi_pool_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autocast/ps_roi_pool_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14 aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autocast/roi_align_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autocast/roi_align_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14 aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autocast/roi_pool_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autocast/roi_pool_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14 aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autograd/deform_conv2d_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autograd/deform_conv2d_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14 In file included from /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include/torch/autograd.h:4:0, from /torchvision/torchvision/csrc/ops/autograd/deform_conv2d_kernel.cpp:3: /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::DeformConv2dBackwardFunction; torch::autograd::variable_list = std::vector]': /torchvision/torchvision/csrc/ops/autograd/deform_conv2d_kernel.cpp:266:1: required from here /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] if (num_outputs > num_forward_inputs) {
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::DeformConv2dFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/deform_conv2d_kernel.cpp:266:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autograd/ps_roi_align_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autograd/ps_roi_align_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include/torch/autograd.h:4:0,
                 from /torchvision/torchvision/csrc/ops/autograd/ps_roi_align_kernel.cpp:3:
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::PSROIAlignBackwardFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/ps_roi_align_kernel.cpp:167:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::PSROIAlignFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/ps_roi_align_kernel.cpp:167:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autograd/ps_roi_pool_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autograd/ps_roi_pool_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include/torch/autograd.h:4:0,
                 from /torchvision/torchvision/csrc/ops/autograd/ps_roi_pool_kernel.cpp:3:
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::PSROIPoolBackwardFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/ps_roi_pool_kernel.cpp:152:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::PSROIPoolFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/ps_roi_pool_kernel.cpp:152:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autograd/roi_align_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autograd/roi_align_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include/torch/autograd.h:4:0,
                 from /torchvision/torchvision/csrc/ops/autograd/roi_align_kernel.cpp:3:
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::ROIAlignBackwardFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/roi_align_kernel.cpp:167:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::ROIAlignFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/roi_align_kernel.cpp:167:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/autograd/roi_pool_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/autograd/roi_pool_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include/torch/autograd.h:4:0,
                 from /torchvision/torchvision/csrc/ops/autograd/roi_pool_kernel.cpp:3:
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::ROIPoolBackwardFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/roi_pool_kernel.cpp:152:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h: In instantiation of 'torch::autograd::variable_list torch::autograd::CppNode<T>::apply(torch::autograd::variable_list&&) [with T = vision::ops::{anonymous}::ROIPoolFunction; torch::autograd::variable_list = std::vector<at::Tensor>]':
/torchvision/torchvision/csrc/ops/autograd/roi_pool_kernel.cpp:152:1:   required from here
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:311:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs > num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:323:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (num_outputs != num_forward_inputs) {
       ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/autograd/custom_function.h:334:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < num_outputs; ++i) {
                   ~~^~~~~~~~~~~~~
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/cpu/deform_conv2d_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu/deform_conv2d_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/cpu/interpolate_aa_kernels.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu/interpolate_aa_kernels.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
In file included from /torchvision/torchvision/csrc/ops/cpu/interpolate_aa_kernels.cpp:4:0:
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/native/UpSample.h:152:31: warning: 'std::array<long int, 5> at::native::upsample_3d_common_check(c10::IntArrayRef, c10::IntArrayRef)' defined but not used [-Wunused-function]
 static std::array<int64_t, 5> upsample_3d_common_check(IntArrayRef input_size, IntArrayRef output_size) {
                               ^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/native/UpSample.h:88:31: warning: 'std::array<long int, 3> at::native::upsample_1d_common_check(c10::IntArrayRef, c10::IntArrayRef)' defined but not used [-Wunused-function]
 static std::array<int64_t, 3> upsample_1d_common_check(IntArrayRef input_size, IntArrayRef output_size) {
                               ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /torchvision/torchvision/csrc/ops/cpu/interpolate_aa_kernels.cpp:2:0:
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/native/IndexingUtils.h:101:1: warning: 'std::tuple<at::Tensor, std::vector<at::Tensor, std::allocator<at::Tensor> > > at::native::transposeToFront(at::Tensor, at::TensorList)' defined but not used [-Wunused-function]
 transposeToFront(Tensor self, TensorList indices) {
 ^~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/native/IndexingUtils.h:83:13: warning: 'bool at::native::hasContiguousSubspace(at::TensorList)' defined but not used [-Wunused-function]
 static bool hasContiguousSubspace(TensorList tl) {
             ^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/native/IndexingUtils.h:54:13: warning: 'void at::native::checkIndexTensorTypes(torch::List<c10::optional<at::Tensor> >&)' defined but not used [-Wunused-function]
 static void checkIndexTensorTypes(const torch::List<c10::optional<Tensor>>& indices) {
             ^~~~~~~~~~~~~~~~~~~~~
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/native/IndexingUtils.h:19:28: warning: 'std::vector<at::Tensor> at::native::expandTensors(const at::Tensor&, torch::List<c10::optional<at::Tensor> >&)' defined but not used [-Wunused-function]
 static std::vector<Tensor> expandTensors(const Tensor & self, const torch::List<c10::optional<Tensor>>& indices) {
                            ^~~~~~~~~~~~~
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/cpu/nms_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu/nms_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/cpu/ps_roi_align_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu/ps_roi_align_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/cpu/ps_roi_pool_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu/ps_roi_pool_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/cpu/roi_align_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu/roi_align_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DWITH_CUDA -I/torchvision/torchvision/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda-10.2/include -I/usr/include/python3.6m -c /torchvision/torchvision/csrc/ops/cpu/roi_pool_kernel.cpp -o build/temp.linux-aarch64-3.6/torchvision/torchvision/csrc/ops/cpu/roi_pool_kernel.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
Traceback (most recent call last):
  File "setup.py", line 488, in <module>
    'clean': clean,
  File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install.py", line 116, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/usr/lib/python3.6/distutils/command/install_lib.py", line 109, in build
    self.run_command('build_ext')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/usr/local/lib/python3.6/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 709, in build_extensions
    build_ext.build_extensions(self)
  File "/usr/local/lib/python3.6/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 457, in unix_wrap_single_compile
    cflags = unix_cuda_flags(cflags)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 424, in unix_cuda_flags
    cflags + _get_cuda_arch_flags(cflags))
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py", line 1575, in _get_cuda_arch_flags
    raise ValueError(f"Unknown CUDA arch ({arch}) or GPU not supported")
ValueError: Unknown CUDA arch () or GPU not supported
The command '/bin/sh -c git clone -b ${TORCHVISION_VERSION} https://github.com/pytorch/vision torchvision &&     cd torchvision &&     python3 setup.py install &&     cd ../ &&     rm -rf torchvision' returned a non-zero code: 1
root@tegra-ubuntu:/usr/sdcard/jetson-containers# 

jonpol01 commented 2 years ago

@dusty-nv oh right sorry for the delay, but I think I have the same build logs with @kevin00730 and also I already moveon onto just using the docker_build_ros.sh --distro foxy --with-slam instead since I need Pytorch + Ros2 than anything with Tensorflow atm.

dusty-nv commented 2 years ago

Thanks @kevin00730 @jonpol01, just pushed a fix for this to https://github.com/dusty-nv/jetson-containers/commit/159acdc69c6f195e901c1c8aa9c0c8dcee3740d8

Can you try pulling the latest from master and trying again?

jonpol01 commented 2 years ago

@dusty-nv Hi, thanks that was an easy fix.. I coulve have got that if I just looked to it more.

anyway tried docker_build_ml.sh pytorch and got this at the end..

it is not compatible with this Python Found link https://files.pythonhosted.org/packages/85/e5/74fcd803fe3a32e8bed53d51889313089124c3ab8fa2a6eeab5cbfba29ab/cmake-3.22.4.tar.gz#sha256=d408349597bf3b9f3ad0364aaed8c55f50dfa011b396d299a7cabf3c799510af (from https://pypi.org/simple/cmake/), version: 3.22.4 Using version 3.22.3 (newest of versions: 3.22.3) Starting new HTTPS connection (1): files.pythonhosted.org https://files.pythonhosted.org:443 "GET /packages/8b/76/6ac2819c82af2d37ec123e56ab67396aaf76652a60858d9e83e8db787fe1/cmake-3.22.3.tar.gz HTTP/1.1" 200 31557 Downloading https://files.pythonhosted.org/packages/8b/76/6ac2819c82af2d37ec123e56ab67396aaf76652a60858d9e83e8db787fe1/cmake-3.22.3.tar.gz Downloading from URL https://files.pythonhosted.org/packages/8b/76/6ac2819c82af2d37ec123e56ab67396aaf76652a60858d9e83e8db787fe1/cmake-3.22.3.tar.gz#sha256=6e1d1991775915dac047af851f2feec7eb35ab4b4610d3e7fd94d93bce8faf58 (from https://pypi.org/simple/cmake/) Running setup.py (path:/tmp/pip-build-16c13zqc/cmake/setup.py) egg_info for package cmake Running command python setup.py egg_info Traceback (most recent call last): File "", line 1, in File "/tmp/pip-build-16c13zqc/cmake/setup.py", line 8, in from skbuild import setup ModuleNotFoundError: No module named 'skbuild' Cleaning up... Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-16c13zqc/cmake/ Exception information: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 342, in run requirement_set.prepare_files(finder) File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files ignore_dependencies=self.ignore_dependencies)) File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 634, in _prepare_file abstract_dist.prep_for_dist() File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 129, in prep_for_dist self.req_to_install.run_egg_info() File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 439, in run_egg_info command_desc='python setup.py egg_info') File "/usr/lib/python3/dist-packages/pip/utils/init.py", line 725, in call_subprocess % (command_desc, proc.returncode, cwd)) pip.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-16c13zqc/cmake/ The command '/bin/sh -c pip3 install --upgrade --no-cache-dir --verbose cmake==3.22.3' returned a non-zero code: 1

Prior to that plenty of "is not compatible with this Python" came up and skipped most of the command. I believe this doesnt even started building stuffs and ended on Step 16.

reading L4T version from /etc/nv_tegra_release L4T BSP Version: L4T R32.6.1 l4t-base image: nvcr.io/nvidia/l4t-base:r32.6.1 selecting OpenCV for L4T R32.6.1... OPENCV_URL=https://nvidia.box.com/shared/static/5v89u6g5rb62fpz4lh0rz531ajo2t5ef.gz OPENCV_DEB=OpenCV-4.5.0-aarch64.tar.gz Python3 version: 3.6 building PyTorch torch-1.9.0-cp36-cp36m-linux_aarch64.whl, torchvision v0.10.0, torchaudio v0.9.0, cuda arch 5.3;6.2;7.2 Building l4t-pytorch:r32.6.1-pth1.9-py3 container... Sending build context to Docker daemon 184.3kB Step 1/26 : ARG BASE_IMAGE=nvcr.io/nvidia/l4t-base:r32.4.4 Step 2/26 : FROM ${BASE_IMAGE} ---> 1735192d3d51 Step 3/26 : ENV DEBIAN_FRONTEND=noninteractive ---> Using cache ---> 85597bf48657 Step 4/26 : RUN apt-get update && apt-get install -y --no-install-recommends python3-pip python3-dev libopenblas-dev libopenmpi-dev openmpi-bin openmpi-common gfortran libomp-dev && rm -rf /var/lib/apt/lists/ && apt-get clean ---> Using cache ---> 90cc60fa245c Step 5/26 : RUN pip3 install --no-cache-dir setuptools Cython wheel ---> Using cache ---> b2935c419e7f Step 6/26 : RUN pip3 install --no-cache-dir --verbose numpy ---> Using cache ---> 4c1a708422d6 Step 7/26 : ARG PYTORCH_URL=https://nvidia.box.com/shared/static/lufbgr3xu2uha40cs9ryq1zn4kxsnogl.whl ---> Using cache ---> 69273822a16a Step 8/26 : ARG PYTORCH_WHL=torch-1.2.0-cp36-cp36m-linux_aarch64.whl ---> Using cache ---> 38053b68f233 Step 9/26 : RUN wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate ${PYTORCH_URL} -O ${PYTORCH_WHL} && pip3 install --no-cache-dir --verbose ${PYTORCH_WHL} && rm ${PYTORCH_WHL} ---> Using cache ---> 9c9bfc177980 Step 10/26 : ARG TORCHVISION_VERSION=v0.4.0 ---> Using cache ---> 729c813ccf7c Step 11/26 : ARG TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2;8.7" ---> Using cache ---> 0b1cf6b073bd Step 12/26 : RUN printenv && echo "torchvision version = $TORCHVISION_VERSION" && echo "TORCH_CUDA_ARCH_LIST = $TORCH_CUDA_ARCH_LIST" ---> Using cache ---> c041fcfd3bcb Step 13/26 : RUN apt-get update && apt-get install -y --no-install-recommends git build-essential libjpeg-dev zlib1g-dev && rm -rf /var/lib/apt/lists/ && apt-get clean ---> Using cache ---> 47bdf95b75f1 Step 14/26 : RUN git clone -b ${TORCHVISION_VERSION} https://github.com/pytorch/vision torchvision && cd torchvision && python3 setup.py install && cd ../ && rm -rf torchvision ---> Using cache ---> 61f94defa33c Step 15/26 : RUN apt-get update && apt-get install -y --no-install-recommends software-properties-common apt-transport-https ca-certificates gnupg lsb-release && rm -rf /var/lib/apt/lists/* && apt-get clean ---> Using cache ---> 737df89eb3d7 Step 16/26 : RUN pip3 install --upgrade --no-cache-dir --verbose cmake==3.22.3 ---> Running in 47dba072d800

I have only been doin ROS2 and jupyterlab added on the ROS2 docker, so I think my Host (Xavier NX) is still quite default.

arpaterson commented 2 years ago

I just experienced this 'no module named skbuild' error too. on nano, JP r32.5.1. I updated jetson-containers and am trying again. Should this be its own issue?

gonzarub commented 2 years ago

EDIT: Code markup

I've been struggling trying to build the full ML container (PyTorch1.x + TF2.7), and one of the first issues I fixed was the "no module named skbuild". I did the following (1) to fix it from Dockerfile.pytorch :

# (2) https://stackoverflow.com/a/66057286
ENV OPENBLAS_CORETYPE=ARMV8

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
            python3-pip \
          python3-dev \
      libopenblas-dev \
      libopenmpi-dev \
        openmpi-bin \
        openmpi-common \
      gfortran \
      libomp-dev \
    && rm -rf /var/lib/apt/lists/* \
   && apt-get clean

# (1) Insert this:
RUN pip3 install --upgrade pip

I figured (1) after reading this: https://github.com/opencv/opencv-python#frequently-asked-questions

Notice fix (2) is not related to this, but helped to fix another error that would hit later, explained by the stackoverflow link in the comment.

Hope that helps

Ruben

arpaterson commented 2 years ago

Thanks @gonzarub. That allows -pth1.9-py3 and -pth1.10-py3 images to be built. It fails to build 1.11 with a code 8 from wget.

Step 11/28 : RUN wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate ${PYTORCH_URL} -O ${PYTORCH_WHL} &&     pip3 install --no-cache-dir --verbose ${PYTORCH_WHL} &&     rm ${PYTORCH_WHL}
 ---> Running in 3a85bcdea451
The command '/bin/sh -c wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate ${PYTORCH_URL} -O ${PYTORCH_WHL} &&     pip3 install --no-cache-dir --verbose ${PYTORCH_WHL} &&     rm ${PYTORCH_WHL}' returned a non-zero code: 8

@dusty-nv I guess the URL is wrong in docker_build_ml.sh, L101. Opening it in a browser gives a 404 not found error.


        # PyTorch v1.11.0
        build_pytorch "https://developer.download.nvidia.com/compute/redist/jp/v461/pytorch/torch-1.11.0a0+17540c5-cp36-cp36m-linux_aarch64.whl" \
                    "torch-1.11.0a0+17540c5-cp36-cp36m-linux_aarch64.whl" \
                    "l4t-pytorch:r$L4T_VERSION-pth1.11-py3" \
                    "v0.11.3" \
                    "v0.10.2"

gonzarub commented 2 years ago

@arpaterson I also found that one. If you look carefully to the single whl file found in the following URL, it is not the same referenced in the build script: https://developer.download.nvidia.com/compute/redist/jp/v461/pytorch/

Of course, I replaced it, and I got to do some more fixes to do [NOTE 1] until I gave up with the master branch. I'm not on PyTorch, and I only wanted a working ML container, so ended up using the last stable branch for L4T 32.7.1 (which is my current version).

I have to say, even that branch required a small fix in the docker_base.sh script.

[NOTE 1] This is what I did before giving up with the master branch:

After changing that path to the torch whl file, I found this link saying that PyTorch 1.11 requires Python 3.7, so I stopped trying this version.
Back to PyTorch 1.10, I detected an invalid apt package (libopenmpi3 had to be replaced with libopenmpi-dev).

And finally, when I thought it was going to work, I got a bunch of Cython-related warnings and an error while building cupy.

  aarch64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE\=2 build/temp.linux-aarch64-3.6/cupy/cudnn.o -L/usr/local/cuda/lib64 -lcudnn -o build/lib.linux-aarch64-3.6/cupy/cudnn.cpython-36m-aarch64-linux-gnu.so -Wl,--disable-new-dtags,-rpath,/usr/local/cuda/lib64
  ^[[91mwarning: cupy/cuda/memory.pyx:119:4: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:495:9: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:539:9: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:697:9: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:713:9: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:728:9: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:739:9: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:754:10: Unreachable code
  warning: cupy_backends/cuda/api/runtime.pyx:773:4: Unreachable code
  warning: cupy_backends/cuda/libs/nvrtc.pyx:87:9: Unreachable code
  warning: cupy_backends/cuda/libs/nvrtc.pyx:183:9: Unreachable code
  ^[[0m^[[91mTraceback (most recent call last):
    File "setup.py", line 149, in <module>
      cmdclass={'build_ext': cupy_builder._command.custom_build_ext},
    File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
 .....
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/local/cuda/bin/nvcc', '-D_FORCE_INLINES=1', '-DCUPY_CUB_VERSION_CODE=100800', '-DCUPY_JITIFY_VERSION_CODE=4a37de0', '-I/cupy/cupy/_core/include/cupy/cub', '-I/cupy/cupy/_core/include', '-I/us\r/local/cuda/include', '-c', 'cupy/cuda/cupy_cub.cu', '-o', 'build/temp.device_objects/cupy/cuda/cupy_cub.cu.o', '--generate-code=arch=compute_53,code=sm_53', '--generate-code=arch=compute_62,code=sm_62', '--generate-code=arch=compute_72,\code=sm_72', '--generate-code=arch=compute_87,code=sm_87', '-O2', '--compiler-options="-fPIC"', '--std=c++11', '-Xcompiler=-fno-gnu-unique']' returned non-zero exit status 1.

YuJuChiu commented 1 year ago

@gonzarub Hello, did you manage the issue? I face the exact same problem as yours

dusty-nv / jetson-containers

ValueError: Unknown CUDA arch () or GPU not supported #140