V2AI / Det3D

World's first general purpose 3D object detection codebse.
https://arxiv.org/abs/1908.09492
Apache License 2.0
1.48k stars 299 forks source link

Cuda execution failed with error 2 #155

Open tuan97anton opened 2 years ago

tuan97anton commented 2 years ago

I run : ./tools/scripts/train.sh train1 I get error: /spconv/src/spconv/indice.cu 125 cuda execution failed with error 2

My PC have only 1 GPU NVIDIA GeForce RTX 2080 SUPER - 8GB My software : CUDA Version 10.0.130 cudnn : 7603 pytorch : 1.3.0 spconv : 1.0 Cmake: cmake version 3.21.3 ninjia: 1.9.0 Python 3.6.8

I installed without docker or with docker , I got the same error /spconv/src/spconv/indice.cu 125 cuda execution failed with error 2 DO you have idea to resolve error above ? Thank you !

tuan97anton commented 2 years ago

My Docker that I installes Det3D, I use extention Remote Development of VisualCode : link

  1. Dockerfile
    
    FROM pytorch/pytorch:1.1.0-cuda10.0-cudnn7.5-devel
    RUN apt-get update && apt-get install -y libboost-all-dev\
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*
    RUN pip install --upgrade pip

RUN pip install Pillow==6.1 RUN apt remove cmake RUN pip install cmake --upgrade RUN pip install setuptools==39.1.0 RUN pip install torch==1.3.0 torchvision==0.4.1

spconv

WORKDIR /root RUN git clone https://github.com/poodarchu/spconv.git --recursive /spconv RUN cd /spconv && python setup.py bdist_wheel RUN cd /spconv/dist && pip install *

nuscenes-devkit

RUN pip install nuscenes-devkit

Det3D

RUN git clone https://github.com/poodarchu/Det3D.git /Det3D WORKDIR /Det3D RUN pip install -r requirements.txt

RUN python setup.py build develop

RUN apt-get update && apt-get install -y libgl1 libgtk2.0-dev

2. devcontainer.json

{ "name": "det3d", "build": { "dockerfile": "Dockerfile", "context": "..", },

// Set *default* container specific settings.json values on container create.
"settings": { 
    "terminal.integrated.profiles.linux": {
        "bash": {
            "path": "/bin/bash"
        }
    },
    "terminal.integrated.defaultProfile.linux": "bash",
    "python.pythonPath": "/usr/local/bin/python",
    "python.languageServer": "Pylance",
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": true,
    "python.formatting.autopep8Path": "/usr/local/py-utils/bin/autopep8",
    "python.formatting.blackPath": "/usr/local/py-utils/bin/black",
    "python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
    "python.linting.banditPath": "/usr/local/py-utils/bin/bandit",
    "python.linting.flake8Path": "/usr/local/py-utils/bin/flake8",
    "python.linting.mypyPath": "/usr/local/py-utils/bin/mypy",
    "python.linting.pycodestylePath": "/usr/local/py-utils/bin/pycodestyle",
    "python.linting.pydocstylePath": "/usr/local/py-utils/bin/pydocstyle",
    "python.linting.pylintPath": "/usr/local/py-utils/bin/pylint"
},

"runArgs": [
    "--name", "my_det3d",
    "--gpus=all", 
    "--shm-size=5G",
    "-v", "/home/ub18-2/Documents/kitti/:/workspaces/my_det3d/data_kitti/",
    "-v", "/home/ub18-2/Documents/nuscenes/v1.0-mini/:/workspaces/my_det3d/data_nusc/"
],

"extensions": [
    "ms-python.python",
    "ms-python.vscode-pylance"
],

}

poodarchu commented 2 years ago

can you try to compile with pytorch 1.6+?

www04576 commented 2 years ago

i got the same problem. have you solved it? thank you.

ShaohuaDong2021 commented 1 year ago

Same question, have you solved it?

lacie-life commented 8 months ago

i have the same problem. have you solved it? thank you.

Edited: I fixed this problem by using spconv 1.2.1