search

nv-nguyen / template-pose

[CVPR 2022] Pytorch implementation of "Templates for 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions" paper
https://nv-nguyen.github.io/template-pose/
178 stars 13 forks source link

env installation #27

Closed monajalal closed 7 months ago

monajalal commented 1 year ago

Following all versions you suggestion + installing Jupyter Lab for running the colab notebook locally on my machine, I get this error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[8], line 2
      1 with torch.no_grad():
----> 2   feat_query = model(query_tensor.to(device))
      4   # get template features by batch
      5   batch_size = 8

File ~/anaconda3/envs/template/lib/python3.8/site-packages/torch/nn/modules/module.py:1051, in Module._call_impl(self, *input, **kwargs)
   1047 # If we don't have any hooks, we want to skip the rest of the logic in
   1048 # this function, and just call forward.
   1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051     return forward_call(*input, **kwargs)
   1052 # Do not call functions when jit is used
   1053 full_backward_hooks, non_full_backward_hooks = [], []

File ~/template-pose/src/model/network.py:60, in FeatureExtractor.forward(self, x)
     59 def forward(self, x):
---> 60     feat = self.backbone(x)
     61     feat = self.projector(feat)
     62     # feat = F.normalize(feat, dim=1)

File ~/anaconda3/envs/template/lib/python3.8/site-packages/torch/nn/modules/module.py:1051, in Module._call_impl(self, *input, **kwargs)
   1047 # If we don't have any hooks, we want to skip the rest of the logic in
   1048 # this function, and just call forward.
   1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051     return forward_call(*input, **kwargs)
   1052 # Do not call functions when jit is used
   1053 full_backward_hooks, non_full_backward_hooks = [], []

File ~/template-pose/src/model/resnet.py:137, in ResNet.forward(self, x)
    135 def forward(self, x):
    136     out = self.conv1(x)
--> 137     out = self.bn1(out)
    138     out = self.relu(out)
    139     if self.use_avg_pooling_and_fc:

File ~/anaconda3/envs/template/lib/python3.8/site-packages/torch/nn/modules/module.py:1051, in Module._call_impl(self, *input, **kwargs)
   1047 # If we don't have any hooks, we want to skip the rest of the logic in
   1048 # this function, and just call forward.
   1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051     return forward_call(*input, **kwargs)
   1052 # Do not call functions when jit is used
   1053 full_backward_hooks, non_full_backward_hooks = [], []

File ~/anaconda3/envs/template/lib/python3.8/site-packages/torch/nn/modules/batchnorm.py:147, in _BatchNorm.forward(self, input)
    144 if self.training and self.track_running_stats:
    145     # TODO: if statement only here to tell the jit to skip emitting this when it is None
    146     if self.num_batches_tracked is not None:  # type: ignore[has-type]
--> 147         self.num_batches_tracked = self.num_batches_tracked + 1  # type: ignore[has-type]
    148         if self.momentum is None:  # use cumulative moving average
    149             exponential_average_factor = 1.0 / float(self.num_batches_tracked)

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
(template) mona@ada:~/template-pose$ conda list
# packages in environment at /home/mona/anaconda3/envs/template:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    anaconda
_openmp_mutex             5.1                       1_gnu    anaconda
absl-py                   0.13.0                   pypi_0    pypi
aiohttp                   3.7.4.post0              pypi_0    pypi
antlr4-python3-runtime    4.9.3                    pypi_0    pypi
anyio                     3.7.1              pyhd8ed1ab_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0           py38h0a891b7_2    conda-forge
argparse                  1.4.0                    pypi_0    pypi
asttokens                 2.0.5              pyhd3eb1b0_0    anaconda
async-timeout             3.0.1                    pypi_0    pypi
attrs                     21.2.0                   pypi_0    pypi
babel                     2.13.1             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyhd3eb1b0_0    anaconda
beautifulsoup4            4.10.0                   pypi_0    pypi
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blenderproc               2.5.0                    pypi_0    pypi
brotlipy                  0.7.0           py38h27cfd23_1003    anaconda
bzip2                     1.0.8                h7b6447c_0    anaconda
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cachetools                4.2.2                    pypi_0    pypi
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cffi                      1.14.5           py38h261ae71_0    anaconda
chardet                   4.0.0           py38h06a4308_1003    anaconda
click                     8.0.1                    pypi_0    pypi
cmake                     3.21.3                   pypi_0    pypi
colorama                  0.4.4                    pypi_0    pypi
commonmark                0.9.1                    pypi_0    pypi
conda-package-handling    1.7.3            py38h27cfd23_1    anaconda
configparser              5.0.2                    pypi_0    pypi
cryptography              3.4.7            py38hd23ed53_0    anaconda
cycler                    0.10.0                   pypi_0    pypi
cython                    0.29.24                  pypi_0    pypi
debugpy                   1.5.1            py38h295c915_0    anaconda
decorator                 5.1.1              pyhd3eb1b0_0    anaconda
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
docker-pycreds            0.4.0                    pypi_0    pypi
easydict                  1.9                      pypi_0    pypi
entrypoints               0.4              py38h06a4308_0    anaconda
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 0.8.2                    pypi_0    pypi
expat                     2.2.10               he6710b0_2    anaconda
filelock                  3.6.0                    pypi_0    pypi
fire                      0.5.0                    pypi_0    pypi
freetype-py               2.2.0                    pypi_0    pypi
fsspec                    2021.6.1                 pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
fvcore                    0.1.5.post20211023          pypi_0    pypi
gdown                     4.4.0                    pypi_0    pypi
gitdb                     4.0.7                    pypi_0    pypi
gitpython                 3.1.18                   pypi_0    pypi
glumpy                    1.2.0                    pypi_0    pypi
google-api-core           2.10.1                   pypi_0    pypi
google-api-python-client  2.65.0                   pypi_0    pypi
google-auth               2.17.2                   pypi_0    pypi
google-auth-httplib2      0.1.0                    pypi_0    pypi
google-auth-oauthlib      1.0.0                    pypi_0    pypi
googleapis-common-protos  1.56.4                   pypi_0    pypi
grpcio                    1.53.0                   pypi_0    pypi
h5py                      3.6.0                    pypi_0    pypi
httplib2                  0.21.0                   pypi_0    pypi
huggingface-hub           0.13.4                   pypi_0    pypi
hydra-core                1.3.2                    pypi_0    pypi
idna                      2.10               pyhd3eb1b0_0    anaconda
imageio                   2.9.0                    pypi_0    pypi
importlib-resources       5.2.0                    pypi_0    pypi
importlib_resources       6.1.1              pyhd8ed1ab_0    conda-forge
iopath                    0.1.9                    pypi_0    pypi
ipykernel                 6.15.2           py38h06a4308_0    anaconda
ipython                   8.1.0                    pypi_0    pypi
ipython_genutils          0.2.0                      py_1    conda-forge
jedi                      0.18.1           py38h06a4308_1    anaconda
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.0.1                    pypi_0    pypi
json5                     0.9.14             pyhd8ed1ab_0    conda-forge
jsonschema                4.19.2             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.7.1           pyhd8ed1ab_0    conda-forge
jupyter_client            7.3.5            py38h06a4308_0    anaconda
jupyter_core              4.11.2           py38h06a4308_0    anaconda
jupyter_server            1.23.4           py38h06a4308_0  
jupyterlab                3.5.0              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.10.3             pyhd3eb1b0_1  
kiwisolver                1.3.1                    pypi_0    pypi
krb5                      1.18.2               h173b8e3_0    anaconda
ld_impl_linux-64          2.38                 h1181459_1    anaconda
libcurl                   7.71.1               h20c2e04_1    anaconda
libedit                   3.1.20191231         h14c3975_1    anaconda
libffi                    3.3                  he6710b0_2    anaconda
libgcc-ng                 11.2.0               h1234567_1    anaconda
libgomp                   11.2.0               h1234567_1    anaconda
libsodium                 1.0.18               h7b6447c_0    anaconda
libssh2                   1.9.0                h1ba5d50_1    anaconda
libstdcxx-ng              11.2.0               h1234567_1    anaconda
libuv                     1.40.0               h7b6447c_0    anaconda
lightly                   1.1.15                   pypi_0    pypi
lightly-utils             0.0.1                    pypi_0    pypi
lightning-utilities       0.3.0                    pypi_0    pypi
llvmlite                  0.38.0                   pypi_0    pypi
lz4-c                     1.9.2                heb0550a_3    anaconda
markdown                  3.3.4                    pypi_0    pypi
markupsafe                2.1.1            py38h7f8727e_0  
mathutils                 2.81.2                   pypi_0    pypi
matplotlib                3.4.2                    pypi_0    pypi
matplotlib-inline         0.1.3                    pypi_0    pypi
meshio                    5.3.0                    pypi_0    pypi
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
multidict                 5.1.0                    pypi_0    pypi
nbclassic                 1.0.0              pyhb4ecaf3_1    conda-forge
nbclient                  0.5.13             pyhd8ed1ab_0    conda-forge
nbconvert                 7.11.0             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.11.0             pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.11.0             pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
ncurses                   6.3                  h5eee18b_3    anaconda
nest-asyncio              1.5.5            py38h06a4308_0    anaconda
networkx                  2.6.2                    pypi_0    pypi
nibabel                   3.2.1                    pypi_0    pypi
notebook                  6.5.6              pyha770c72_0    conda-forge
notebook-shim             0.2.3              pyhd8ed1ab_0    conda-forge
numba                     0.55.1                   pypi_0    pypi
numpy                     1.20.3                   pypi_0    pypi
oauth2client              4.1.3                    pypi_0    pypi
oauthlib                  3.1.1                    pypi_0    pypi
omegaconf                 2.3.0                    pypi_0    pypi
opencv-python             4.5.2.52                 pypi_0    pypi
openssl                   1.1.1w               h7f8727e_0  
packaging                 20.9                     pypi_0    pypi
pandas                    1.3.1                    pypi_0    pypi
pandoc                    2.19.2               ha770c72_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd3eb1b0_0    anaconda
pathtools                 0.1.2                    pypi_0    pypi
pexpect                   4.8.0              pyhd3eb1b0_3    anaconda
pickleshare               0.7.5           pyhd3eb1b0_1003    anaconda
pillow                    8.2.0                    pypi_0    pypi
pip                       22.2.2           py38h06a4308_0    anaconda
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
plyfile                   0.7.4                    pypi_0    pypi
portalocker               2.3.2                    pypi_0    pypi
progressbar               2.5                      pypi_0    pypi
prometheus_client         0.18.0             pyhd8ed1ab_0    conda-forge
promise                   2.3                      pypi_0    pypi
prompt-toolkit            3.0.28                   pypi_0    pypi
prompt_toolkit            3.0.40               hd8ed1ab_0    conda-forge
protobuf                  3.20.1                   pypi_0    pypi
psutil                    5.8.0                    pypi_0    pypi
ptyprocess                0.7.0              pyhd3eb1b0_2    anaconda
pure_eval                 0.2.2              pyhd3eb1b0_0    anaconda
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pybind11                  2.10.1                   pypi_0    pypi
pycosat                   0.6.3            py38h7b6447c_1    anaconda
pycparser                 2.20                       py_2    anaconda
pydeprecate               0.3.0                    pypi_0    pypi
pyglet                    1.5.16                   pypi_0    pypi
pygments                  2.11.2             pyhd3eb1b0_0    anaconda
pyopengl                  3.1.0                    pypi_0    pypi
pyopenssl                 20.0.1             pyhd3eb1b0_1    anaconda
pyparsing                 2.4.7                    pypi_0    pypi
pypng                     0.0.21                   pypi_0    pypi
pyrender                  0.1.45                   pypi_0    pypi
pysocks                   1.7.1            py38h06a4308_0    anaconda
python                    3.8.13               haa1d7c7_1    anaconda
python-dateutil           2.8.1                    pypi_0    pypi
python-fastjsonschema     2.18.1             pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytorch-lightning         1.8.1                    pypi_0    pypi
pytorch3d                 0.7.5                     dev_0    <develop>
pytz                      2021.1                   pypi_0    pypi
pyyaml                    5.4.1                    pypi_0    pypi
pyzmq                     23.2.0           py38h6a678d5_0    anaconda
readline                  8.2                  h5eee18b_0    anaconda
referencing               0.30.2             pyhd8ed1ab_0    conda-forge
requests                  2.25.1             pyhd3eb1b0_0    anaconda
requests-oauthlib         1.3.0                    pypi_0    pypi
requests-random-user-agent 2022.11.12               pypi_0    pypi
rhash                     1.4.0                h1ba5d50_0    anaconda
rich                      11.1.0                   pypi_0    pypi
rpds-py                   0.10.6           py38hb02cf49_0  
rsa                       4.7.2                    pypi_0    pypi
ruamel-yaml               0.17.16                  pypi_0    pypi
ruamel-yaml-clib          0.2.6                    pypi_0    pypi
ruamel_yaml               0.15.100         py38h27cfd23_0    anaconda
scikit-learn              0.24.2                   pypi_0    pypi
scipy                     1.6.3                    pypi_0    pypi
seaborn                   0.11.2                   pypi_0    pypi
send2trash                1.8.2              pyh41d4057_0    conda-forge
sentry-sdk                1.3.1                    pypi_0    pypi
setuptools                65.5.0           py38h06a4308_0    anaconda
shortuuid                 1.0.1                    pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_1    anaconda
sklearn                   0.0                      pypi_0    pypi
smmap                     4.0.0                    pypi_0    pypi
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.1                    pypi_0    pypi
sqlite                    3.39.3               h5082296_0    anaconda
stack_data                0.2.0              pyhd3eb1b0_0    anaconda
structlog                 21.5.0                   pypi_0    pypi
subprocess32              3.5.4                    pypi_0    pypi
tabulate                  0.8.9                    pypi_0    pypi
tensorboard               2.12.1                   pypi_0    pypi
tensorboard-data-server   0.7.0                    pypi_0    pypi
tensorboard-plugin-wit    1.8.0                    pypi_0    pypi
tensorboardx              2.5.1                    pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
terminado                 0.18.0             pyh0d859eb_0    conda-forge
threadpoolctl             2.2.0                    pypi_0    pypi
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h1ccaba5_0    anaconda
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
torch                     1.9.0                    pypi_0    pypi
torch-tb-profiler         0.4.1                    pypi_0    pypi
torchmetrics              0.11.4                   pypi_0    pypi
torchvision               0.10.0                   pypi_0    pypi
tornado                   6.2              py38h5eee18b_0    anaconda
tqdm                      4.59.0             pyhd3eb1b0_1    anaconda
traitlets                 5.1.1              pyhd3eb1b0_0    anaconda
triangle                  20200424                 pypi_0    pypi
trimesh                   3.9.25                   pypi_0    pypi
ttach                     0.0.3                    pypi_0    pypi
typing-extensions         4.5.0                    pypi_0    pypi
typing_extensions         4.8.0              pyha770c72_0    conda-forge
uritemplate               4.1.1                    pypi_0    pypi
urllib3                   1.26.4             pyhd3eb1b0_0    anaconda
vispy                     0.6.6                    pypi_0    pypi
wandb                     0.12.1                   pypi_0    pypi
wcwidth                   0.2.5              pyhd3eb1b0_0    anaconda
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.6.4              pyhd8ed1ab_0    conda-forge
werkzeug                  2.0.1                    pypi_0    pypi
wget                      3.2                      pypi_0    pypi
wheel                     0.37.1             pyhd3eb1b0_0    anaconda
xz                        5.2.6                h5eee18b_0    anaconda
yacs                      0.1.8                    pypi_0    pypi
yaml                      0.2.5                h7b6447c_0    anaconda
yarl                      1.6.3                    pypi_0    pypi
zeromq                    4.3.4                h2531618_0    anaconda
zipp                      3.5.0                    pypi_0    pypi
zlib                      1.2.13               h5eee18b_0    anaconda
zstd                      1.4.5                h9ceee32_0    anaconda

(template) mona@ada:~$ uname -a
Linux ada 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
(template) mona@ada:~$ lsb_release -a
LSB Version:    core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy
(template) mona@ada:~$ nvidia-smi -a

==============NVSMI LOG==============

Timestamp                                 : Mon Nov 13 14:09:21 2023
Driver Version                            : 535.104.12
CUDA Version                              : 12.2

Attached GPUs                             : 1
GPU 00000000:52:00.0
    Product Name                          : NVIDIA RTX 6000 Ada Generation
    Product Brand                         : NVIDIA RTX
    Product Architecture                  : Ada Lovelace
    Display Mode                          : Enabled
    Display Active                        : Enabled
    Persistence Mode                      : Enabled

(template) mona@ada:~$ nvidia-smi 
Mon Nov 13 14:09:42 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.12             Driver Version: 535.104.12   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:52:00.0  On |                  Off |
| 30%   43C    P8              28W / 300W |   1733MiB / 49140MiB |     14%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      2253      G   /usr/lib/xorg/Xorg                          343MiB |
|    0   N/A  N/A      2443      G   /usr/bin/gnome-shell                        119MiB |
|    0   N/A  N/A      2935      G   ...AAAAAAAACAAAAAAAAAA= --shared-files       56MiB |
|    0   N/A  N/A      3656      G   ...irefox/3290/usr/lib/firefox/firefox      241MiB |
|    0   N/A  N/A      4898      G   ...1503406,14595540716616834367,262144       71MiB |
|    0   N/A  N/A     19786      G   ...ures=SpareRendererForSitePerProcess       34MiB |
|    0   N/A  N/A     21050      C   .../anaconda3/envs/template/bin/python      828MiB |
+---------------------------------------------------------------------------------------+

for this block of code:

with torch.no_grad():
  feat_query = model(query_tensor.to(device))

  # get template features by batch
  batch_size = 8
  list_feat_templates = []
  for start_index in tqdm(range(0, len(list_templates), batch_size)):
    templates = list_templates[start_index:start_index+batch_size]
    feat_template = model(templates.to(device))
    list_feat_templates.append(feat_template)
list_feat_templates = torch.cat(list_feat_templates, dim=0)
(base) mona@ada:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

I restarted the machine and I still have the same problem

(base) mona@ada:~/template-pose$ conda activate template
(template) mona@ada:~/template-pose$ python
Python 3.8.13 (default, Oct 21 2022, 23:50:54) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'1.9.0+cu102'
>>> torch.cuda.is_available()
True
>>> torch.version.cuda
'10.2'
monajalal commented 7 months ago

closing the issue due to no response.