NVIDIAGameWorks / kaolin

A PyTorch Library for Accelerating 3D Deep Learning Research
Apache License 2.0
4.5k stars 562 forks source link

ImportError of kaolin in Colab #597

Closed cpfy closed 2 years ago

cpfy commented 2 years ago

I received ImportError while importing kaolin in Colab. And my question is similar to Issue #363. But it has another kind of 'undefined symbol' error info. This may be an issue with input data format or due to features of kaolin? Anybody can help me here? Thanks.

Here is the details:

Traceback (most recent call last): File "tools/prepare_data/prepare_data_cache.py", line 5, in from datasets import dataset_dict File "./datasets/init.py", line 1, in from .phototourism import PhototourismDataset File "./datasets/phototourism.py", line 16, in from tools.prepare_data.generate_voxel import get_near_far, gen_octree_from_sfm File "./tools/prepare_data/generate_voxel.py", line 13, in from kaolin.ops import spc File "/usr/local/lib/python3.7/dist-packages/kaolin/init.py", line 1, in from . import io File "/usr/local/lib/python3.7/dist-packages/kaolin/io/init.py", line 5, in from . import render File "/usr/local/lib/python3.7/dist-packages/kaolin/io/render.py", line 23, in from ..render.camera import generate_perspective_projection File "/usr/local/lib/python3.7/dist-packages/kaolin/render/init.py", line 2, in from . import mesh File "/usr/local/lib/python3.7/dist-packages/kaolin/render/mesh/init.py", line 1, in from .utils import * File "/usr/local/lib/python3.7/dist-packages/kaolin/render/mesh/utils.py", line 22, in from ... import ops File "/usr/local/lib/python3.7/dist-packages/kaolin/ops/init.py", line 1, in from . import batch File "/usr/local/lib/python3.7/dist-packages/kaolin/ops/batch.py", line 17, in from kaolin import _C ImportError: /usr/local/lib/python3.7/dist-packages/kaolin/_C.so: undefined symbol: _ZNK3c1010TensorImpl36is_contiguous_nondefault_policy_implENS_12MemoryFormatE

Caenorst commented 2 years ago

Hi @cpfy From the look at it, my guess is that you've built Kaolin then changed pytorch version. Can you verify?

cpfy commented 2 years ago

Hi, thanks for the advice! After I dropped the version of torch from 1.12.0 to 1.11.0 and installed kaolin from git clone and manually developed . It works well now! It seems that some dependencies are missing when installing from pip. Unmatched torch version and build order are possible reasons too.

Caenorst commented 2 years ago

Glad you got it working!

I wouldn't recommend to install from pip you will see that our installation guide recommend python setup.py develop for more robust installation.

FavorMylikes commented 2 years ago

@cpfy Could you provide the output of conda list? I wanna check which part is different

cpfy commented 2 years ago

@cpfy Could you provide the output of conda list? I wanna check which part is different

My code is running on Google Colab and here is only the output of pip list (after installation of kaolin)

Hope it can help you~

Package                       Version                      Location
----------------------------- ---------------------------- ------------------------------------------------
absl-py                       1.2.0
addict                        2.4.0
aiobotocore                   2.1.2
aiohttp                       3.8.1
aioitertools                  0.10.0
aiosignal                     1.2.0
alabaster                     0.7.12
albumentations                0.1.12
altair                        4.2.0
anyio                         3.6.1
appdirs                       1.4.4
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arviz                         0.12.1
asgiref                       3.5.2
astor                         0.8.1
astropy                       4.3.1
astunparse                    1.6.3
async-timeout                 4.0.2
asynctest                     0.13.0
atari-py                      0.2.9
atomicwrites                  1.4.1
attrs                         21.4.0
audioread                     2.1.9
autograd                      1.4
Babel                         2.10.3
backcall                      0.2.0
beautifulsoup4                4.6.3
bleach                        5.0.1
blis                          0.7.8
bokeh                         2.3.3
botocore                      1.23.24
branca                        0.5.0
bs4                           0.0.1
CacheControl                  0.12.11
cached-property               1.5.2
cachetools                    4.2.4
catalogue                     2.0.8
certifi                       2022.6.15
cffi                          1.15.1
cftime                        1.6.1
chardet                       3.0.4
charset-normalizer            2.1.0
click                         7.1.2
clikit                        0.6.2
cloudpickle                   1.3.0
cmake                         3.22.5
cmdstanpy                     1.0.4
colorcet                      3.0.0
colorlover                    0.3.0
commonmark                    0.9.1
community                     1.0.0b1
contextlib2                   0.5.5
convertdate                   2.4.0
coverage                      3.7.1
coveralls                     0.5
crashtest                     0.3.1
crcmod                        1.7
croniter                      1.3.5
cufflinks                     0.17.3
cupy-cuda111                  9.4.0
cvxopt                        1.2.7
cvxpy                         1.0.31
cycler                        0.11.0
cymem                         2.0.6
Cython                        0.29.20
daft                          0.0.4
dask                          2.12.0
datascience                   0.10.6
debugpy                       1.0.0
decorator                     4.4.2
deepdiff                      5.8.1
defusedxml                    0.7.1
deprecat                      2.1.1
descartes                     1.1.0
dill                          0.3.5.1
distributed                   1.25.3
dlib                          19.24.0
dm-tree                       0.1.7
dnspython                     2.2.1
docopt                        0.6.2
docutils                      0.17.1
dopamine-rl                   1.0.5
earthengine-api               0.1.317
easydict                      1.9
ecos                          2.0.10
editdistance                  0.5.3
email-validator               1.2.1
en-core-web-sm                3.4.0
entrypoints                   0.4
ephem                         4.1.3
et-xmlfile                    1.1.0
etils                         0.6.0
fa2                           0.3.5
fastai                        2.7.7
fastapi                       0.79.0
fastcore                      1.5.10
fastdownload                  0.0.7
fastdtw                       0.3.4
fastjsonschema                2.16.1
fastprogress                  1.0.3
fastrlock                     0.8
feather-format                0.4.1
filelock                      3.7.1
firebase-admin                4.4.0
fix-yahoo-finance             0.0.22
Flask                         1.1.4
flatbuffers                   2.0
folium                        0.8.3
frozenlist                    1.3.0
fsspec                        2022.1.0
future                        0.18.2
gast                          0.5.3
GDAL                          2.2.2
gdown                         4.4.0
gensim                        3.6.0
geographiclib                 1.52
geopy                         1.17.0
gin-config                    0.5.0
glob2                         0.7
google                        2.0.3
google-api-core               1.31.6
google-api-python-client      1.12.11
google-auth                   1.35.0
google-auth-httplib2          0.0.4
google-auth-oauthlib          0.4.6
google-cloud-bigquery         1.21.0
google-cloud-bigquery-storage 1.1.2
google-cloud-core             1.0.3
google-cloud-datastore        1.8.0
google-cloud-firestore        1.7.0
google-cloud-language         1.2.0
google-cloud-storage          1.18.1
google-cloud-translate        1.5.0
google-colab                  1.0.0
google-pasta                  0.2.0
google-resumable-media        0.4.1
googleapis-common-protos      1.56.4
googledrivedownloader         0.4
graphviz                      0.10.1
greenlet                      1.1.2
grpcio                        1.47.0
gspread                       3.4.2
gspread-dataframe             3.0.8
gym                           0.17.3
h11                           0.13.0
h5py                          3.1.0
HeapDict                      1.0.1
hijri-converter               2.2.4
holidays                      0.14.2
holoviews                     1.14.9
html5lib                      1.0.1
httpimport                    0.5.18
httplib2                      0.17.4
httplib2shim                  0.0.3
httpstan                      4.6.1
httptools                     0.4.0
humanize                      0.5.1
hyperopt                      0.1.2
idna                          2.10
imageio                       2.4.1
imagesize                     1.4.1
imbalanced-learn              0.8.1
imblearn                      0.0
imgaug                        0.2.9
importlib-metadata            4.12.0
importlib-resources           5.9.0
imutils                       0.5.4
inflect                       2.1.0
iniconfig                     1.1.1
intel-openmp                  2022.1.0
intervaltree                  2.1.0
ipykernel                     4.10.1
ipython                       5.5.0
ipython-genutils              0.2.0
ipython-sql                   0.3.9
ipywidgets                    7.7.1
itsdangerous                  2.1.2
jax                           0.3.14
jaxlib                        0.3.14+cuda11.cudnn805
jedi                          0.18.1
jieba                         0.42.1
Jinja2                        3.0.3
jmespath                      0.10.0
joblib                        1.1.0
jpeg4py                       0.1.4
jsonschema                    4.3.3
jupyter                       1.0.0
jupyter-client                5.3.5
jupyter-console               5.2.0
jupyter-core                  4.11.1
jupyterlab-pygments           0.2.2
jupyterlab-widgets            1.1.1
kaggle                        1.5.12
kaolin                        0.11.0                       /path/to/kaolin (In Google Drive)
kapre                         0.3.7
keras                         2.8.0
Keras-Preprocessing           1.1.2
keras-vis                     0.4.1
kiwisolver                    1.4.4
korean-lunar-calendar         0.2.1
kornia                        0.4.1
langcodes                     3.3.0
libclang                      14.0.1
librosa                       0.8.1
lightgbm                      2.2.3
lightning                     2022.7.31
lightning-cloud               0.5.0
llvmlite                      0.34.0
lmdb                          0.99
loguru                        0.6.0
lpips                         0.1.3
LunarCalendar                 0.0.9
lxml                          4.2.6
Markdown                      3.4.1
MarkupSafe                    2.0.1
marshmallow                   3.17.0
matplotlib                    3.2.2
matplotlib-inline             0.1.3
matplotlib-venn               0.11.7
missingno                     0.5.1
mistune                       0.8.4
mizani                        0.6.0
mkl                           2019.0
mlxtend                       0.14.0
more-itertools                8.13.0
moviepy                       0.2.3.5
mpmath                        1.2.1
msgpack                       1.0.4
multidict                     6.0.2
multiprocess                  0.70.13
multitasking                  0.0.11
murmurhash                    1.0.7
music21                       5.5.0
natsort                       5.5.0
nbclient                      0.6.6
nbconvert                     5.6.1
nbformat                      5.4.0
nest-asyncio                  1.5.5
netCDF4                       1.6.0
networkx                      2.6.3
nibabel                       3.0.2
nltk                          3.7
notebook                      5.3.1
numba                         0.51.2
numexpr                       2.8.3
numpy                         1.21.6
oauth2client                  4.1.3
oauthlib                      3.2.0
okgrade                       0.4.3
open3d                        0.12.0
opencv-contrib-python         4.6.0.66
opencv-python                 4.6.0.66
openpyxl                      3.0.10
opt-einsum                    3.3.0
ordered-set                   4.1.0
orjson                        3.7.10
osqp                          0.6.2.post0
packaging                     21.3
palettable                    3.3.0
pandas                        1.3.5
pandas-datareader             0.9.0
pandas-gbq                    0.13.3
pandas-profiling              1.4.1
pandocfilters                 1.5.0
panel                         0.12.1
param                         1.12.2
parso                         0.8.3
pastel                        0.2.1
pathlib                       1.0.1
pathy                         0.6.2
patsy                         0.5.2
pep517                        0.12.0
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        7.1.2
pip                           21.1.3
pip-tools                     6.2.0
plotly                        5.5.0
plotnine                      0.6.0
pluggy                        0.7.1
plyfile                       0.7.4
pooch                         1.6.0
portpicker                    1.3.9
prefetch-generator            1.0.1
preshed                       3.0.6
prettytable                   3.3.0
progressbar2                  3.38.0
prometheus-client             0.14.1
promise                       2.3
prompt-toolkit                1.0.18
prophet                       1.1
protobuf                      3.17.3
psutil                        5.4.8
psycopg2                      2.7.6.1
ptyprocess                    0.7.0
py                            1.11.0
pyarrow                       6.0.1
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycocotools                   2.0.4
pycparser                     2.21
pyct                          0.4.8
pydantic                      1.9.1
pydata-google-auth            1.4.0
pyDeprecate                   0.3.1
pydot                         1.3.0
pydot-ng                      2.0.0
pydotplus                     2.0.2
PyDrive                       1.3.1
pyemd                         0.5.1
pyerfa                        2.0.0.1
pyglet                        1.5.0
Pygments                      2.6.1
pygobject                     3.26.1
PyJWT                         2.4.0
pylev                         1.4.0
pymc3                         3.11.5
PyMeeus                       0.5.11
pymongo                       4.2.0
pymystem3                     0.2.0
PyOpenGL                      3.1.6
pyparsing                     3.0.9
pyrsistent                    0.18.1
pysimdjson                    3.2.0
pysndfile                     1.3.8
PySocks                       1.7.1
pystan                        3.3.0
pytest                        3.6.4
python-apt                    0.0.0
python-chess                  0.23.11
python-dateutil               2.8.2
python-dotenv                 0.20.0
python-louvain                0.16
python-multipart              0.0.5
python-slugify                6.1.2
python-utils                  3.3.3
pytorch-lightning             1.4.8
pytorch-ranger                0.1.1
pytz                          2022.1
pyviz-comms                   2.2.0
PyWavelets                    1.3.0
PyYAML                        6.0
pyzmq                         23.2.0
qdldl                         0.1.5.post2
qtconsole                     5.3.1
QtPy                          2.1.0
regex                         2022.6.2
requests                      2.23.0
requests-oauthlib             1.3.1
resampy                       0.3.1
rich                          12.5.1
rpy2                          3.4.5
rsa                           4.9
s3fs                          2022.1.0
scikit-image                  0.18.3
scikit-learn                  1.0.2
scipy                         1.7.3
screen-resolution-extra       0.0.0
scs                           3.2.0
seaborn                       0.11.2
semver                        2.13.0
Send2Trash                    1.8.0
setuptools                    57.4.0
setuptools-git                1.2
Shapely                       1.8.2
simplegeneric                 0.8.1
six                           1.15.0
sklearn                       0.0
sklearn-pandas                1.8.0
smart-open                    5.2.1
sniffio                       1.2.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
SoundFile                     0.10.3.post1
soupsieve                     2.3.2.post1
spacy                         3.4.1
spacy-legacy                  3.0.9
spacy-loggers                 1.0.3
Sphinx                        1.8.6
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-websupport      1.2.4
SQLAlchemy                    1.4.39
sqlparse                      0.4.2
srsly                         2.4.4
starlette                     0.19.1
starsessions                  1.2.3
statsmodels                   0.10.2
sympy                         1.7.1
tables                        3.7.0
tabulate                      0.8.10
tblib                         1.7.0
tenacity                      8.0.1
tensorboard                   2.8.0
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorflow                    2.8.2+zzzcolab20220719082949
tensorflow-datasets           4.0.1
tensorflow-estimator          2.8.0
tensorflow-gcs-config         2.8.0
tensorflow-hub                0.12.0
tensorflow-io-gcs-filesystem  0.26.0
tensorflow-metadata           1.9.0
tensorflow-probability        0.16.0
termcolor                     1.1.0
terminado                     0.13.3
test-tube                     0.7.5
testpath                      0.6.0
text-unidecode                1.3
textblob                      0.15.3
Theano-PyMC                   1.1.2
thinc                         8.1.0
threadpoolctl                 3.1.0
tifffile                      2021.11.2
tinycss2                      1.1.1
tomli                         2.0.1
toolz                         0.12.0
torch                         1.12.0+cu113
torch-optimizer               0.3.0
torchaudio                    0.12.0+cu113
torchmetrics                  0.7.0
torchsummary                  1.5.1
torchtext                     0.13.0
torchvision                   0.13.0+cu113
tornado                       5.1
tqdm                          4.64.0
traitlets                     5.1.1
trimesh                       3.9.1
tweepy                        3.10.0
typeguard                     2.7.1
typer                         0.4.2
typing-extensions             4.1.1
tzlocal                       1.5.1
ujson                         5.4.0
uritemplate                   3.0.1
urllib3                       1.25.11
usd-core                      22.5.post1
uvicorn                       0.17.6
uvloop                        0.16.0
vega-datasets                 0.9.0
wasabi                        0.9.1
watchgod                      0.8.2
wcwidth                       0.2.5
webargs                       8.2.0
webencodings                  0.5.1
websocket-client              1.3.3
websockets                    10.3
Werkzeug                      1.0.1
wheel                         0.37.1
widgetsnbextension            3.6.1
wordcloud                     1.5.0
wrapt                         1.14.1
xarray                        0.20.2
xarray-einstats               0.2.2
xgboost                       0.90
xkit                          0.0.0
xlrd                          1.1.0
xlwt                          1.3.0
yacs                          0.1.8
yarl                          1.7.2
yellowbrick                   1.4
zict                          2.2.0
zipp                          3.8.1
FavorMylikes commented 2 years ago

I've solved it, it needs nvcc as required

gushengbo commented 1 year ago

I've solved it, it needs nvcc as required

@FavorMylikes hello, could you tell me how to know whether I have nvcc?

FavorMylikes commented 1 year ago

I've solved it, it needs nvcc as required

@FavorMylikes hello, could you tell me how to know whether I have nvcc?

@gushengbo

~> nvcc -V                                                                                                                                                                                                           (kaolin) 
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Wed_Jun__2_19:15:15_PDT_2021
Cuda compilation tools, release 11.4, V11.4.48
Build cuda_11.4.r11.4/compiler.30033411_0
gushengbo commented 1 year ago

@FavorMylikes I am a non-root in the cluster, so when I input nvcc -V, there is no command name"nvcc". That means I don't have nvcc or I don't have access

cpfy commented 1 year ago

@FavorMylikes I am a non-root in the cluster, so when I input nvcc -V, there is no command name"nvcc". That means I don't have nvcc or I don't have access

Are you running the code on Colab or Linux server?

Colab should automatically install and configure the cuda-related environment, and you don't need to worry about it.

If it is on a Linux server. Maybe you can check if there is nvcc file under folder /usr/local/cuda/bin. My guess is that you may not have configured environment variables for CUDA and you need to add this dir to your $PATH.

gushengbo commented 1 year ago

1.kaolin/csrc/ops/conversions/unbatched mcube/unbatched mcube cuda.cu(659):error: morehan one instance of overloaded function "log2" matches the argument list:function "og2( double)"function "log2( float )"argument types are: ( int ) kaolin/csrc/ops/conversions/unbatched mcube/unbatched mcube cuda.cu(660): error: morehan one instance of overloaded function "log2" matches the arqument list:function "loa2( double)"function "Tog2( float)"argument types are: ( int ) kaolin/csrc/ops/conversions/unbatched mcube/unbatched mcube cuda.cu(661): error: more than one instance of overloaded function "log2" matches the argument list:function "og2(double)"function "log2( float )"argument types are: ( int) 3 errors detected in the compilation of "/tmp/tmpxft 00004231 00000000-6 unbatched mcube cuda.cpp1.ii"error: command '/usr/local/cuda/bin/nvcc' failed with exit code 1

2.kaolin/csrc/ops/conversions/mesh to spc/mesh_to_spc_cuda.cu:21:10: fatal error: cub/device/device_scan.cuh: No such file or directory

include <cub/device/device scan .cuh>

error: compilation terminated.

@cpfy yesh, it is on a Linux server. Above are two errors I made when installing. It seems to be I have no nvcc?

cpfy commented 1 year ago

@gushengbo Yes, that's my guess too. If there is already cuda or cuda-X.Y folder in the usr/local directory, it is probably not configured correctly, otherwise you need to install it first

gushengbo commented 1 year ago

@cpfy, ok, thank you. I will have a try!

gushengbo commented 1 year ago

@cpfy, I was able to download it offline(kaolin0.11.0). But when I was executing the program, there was a bug. File "/home/shengbo/ICON-master/lib/net/HGPIFuNet.py",Tine 307,in querypoint feat out = point feat extractor.query(File "/home/shengbo/ICON-master/lib/dataset/PointFeat.py", ine 44,in queryresidues, pts ind,= point to mesh distance( points, self.triangles )File "/home/shengbo/anaconda3/envs/icon/lib/python3.8/site-packages/kaolin/metr ics/tr iangleme.line 75, in point to mesh distancecur dist, cur face idx, cur dist type = UnbatchedTriangleDistanceCuda.apply(File "/home/shengbo/anaconda3/envs/ icon/lib/python3.8/site-packages/kaolin/metrics/triangleme.line 119,in forward C.metrics.unbatched triangle distance forward cudau( timeError: CUDA error: initialization error

That's probably due to nvcc, right?

cpfy commented 1 year ago

Yeah, I think so.

Also you need to strictly refer to the documentation to install kaolin: https://kaolin.readthedocs.io/en/latest/notes/installation.html

gushengbo commented 1 year ago

OK, thank you !