search

isl-org / Open3D-ML

An extension of Open3D to address 3D Machine Learning tasks
Other
1.74k stars 313 forks source link

Input key error in 3d object detection example #433

Open jsohn123 opened 2 years ago

jsohn123 commented 2 years ago

Using master (12/04/2021) + bypassing tensor_requirements to be 2.4.1+ on ubuntu 20.04 = successful data visualization on data reader example using semantic KITTI dataset.

After downloading the KITTI data/label/calib as per dataset python script, I'm running the following example for the point pillars

import os import open3d.ml as _ml3d

import open3d.ml.tf as ml3d

cfg_file = "/content/point_pillars/Open3D-ML/ml3d/configs/pointpillars_kitti.yml" cfg = _ml3d.utils.Config.load_from_file(cfg_file)

model = ml3d.models.PointPillars(cfg.model) cfg.dataset['dataset_path'] = "/content/KITTI_DATASET/KITTI_PTCLOUD_DATA/data_object_velodyne" dataset = ml3d.datasets.KITTI(cfg.dataset.pop('dataset_path', None), cfg.dataset)

pipeline = ml3d.pipelines.ObjectDetection(model, dataset=dataset, device="gpu", **cfg.pipeline)

ckpt_path = '/content/KITTI_DATASET/pointpillars_kitti_202012221652utc/ckpt-12'

pipeline.load_ckpt(ckpt_path=ckpt_path)

test_split = dataset.get_split("test")

data = test_split.get_data(0)

result = pipeline.run_inference(data)

pipeline.run_test() '''

Yields the following error (most of them warnings for not having CUDA):

2021-12-04 21:22:00.545338: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2021-12-04 21:22:00.545360: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. 2021-12-04 21:22:01.339405: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2021-12-04 21:22:01.339424: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. 2021-12-04 21:22:01.908347: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set 2021-12-04 21:22:01.908474: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory 2021-12-04 21:22:01.908485: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303) 2021-12-04 21:22:01.908501: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (dsohn03): /proc/driver/nvidia/version does not exist 2021-12-04 21:22:01.908649: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-12-04 21:22:01.909075: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set INFO - 2021-12-04 21:22:02,104 - object_detection - Restored from /home/daniel.sohn/Desktop/KITTI_DATASET/pointpillars_kitti_202012221652utc/ckpt-12 INFO - 2021-12-04 21:22:02,105 - kitti - Found 7518 pointclouds for test Traceback (most recent call last): File "./demo_ptpillars.py", line 37, in result = pipeline.run_inference(data) File "/content/point_pillars/env/lib/python3.8/site-packages/open3d/_ml3d/tf/pipelines/object_detection.py", line 57, in run_inference results = model(data, training=False) File "/content/point_pillars/env/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1012, in call outputs = call_fn(inputs, *args, **kwargs) File "/content/point_pillars/env/lib/python3.8/site-packages/open3d/_ml3d/tf/models/point_pillars.py", line 142, in call inputs = unpack(inputs[0], inputs[-2]) KeyError: 0

Is this a repeat issue of https://github.com/isl-org/Open3D-ML/issues/408 ?

tensorflow 2.4.4 open3d 0.13.0

Thank you,

sanskar107 commented 2 years ago

Could you please try with the latest version of Open3D (0.14.1) which was released two days ago?

jsohn123 commented 2 years ago

Thanks for the reply. The issue is still there with tf 2.5.2 + open3D 0.14.1 on MASTER branch.

File "/content/point_pillars/env/lib/python3.8/site-packages/open3d/_ml3d/tf/models/point_pillars.py", line 142, in call inputs = unpack(inputs[0], inputs[-2]) KeyError: 0

FYI, I've extracted the lidar datas (3 categories) to the same folder - /content/KITTI_DATASET/KITTI_PTCLOUD_DATA/ contains "data_object_velodyne", "data_object_label_2", and "data_object_calib" on the same level.

Package Version

absl-py 0.15.0 addict 2.4.0 anyio 3.4.0 argon2-cffi 21.1.0 astunparse 1.6.3 attrs 21.2.0 Babel 2.9.1 backcall 0.2.0 bleach 4.1.0 cachetools 4.2.4 certifi 2021.10.8 cffi 1.15.0 charset-normalizer 2.0.9 cycler 0.11.0 debugpy 1.5.1 decorator 5.1.0 defusedxml 0.7.1 deprecation 2.1.0 entrypoints 0.3 flatbuffers 1.12 fonttools 4.28.3 gast 0.4.0 google-auth 2.3.3 google-auth-oauthlib 0.4.6 google-pasta 0.2.0 grpcio 1.34.1 h5py 3.1.0 idna 3.3 importlib-metadata 4.8.2 importlib-resources 5.4.0 ipykernel 6.6.0 ipython 7.30.1 ipython-genutils 0.2.0 ipywidgets 7.6.5 jedi 0.18.1 Jinja2 3.0.3 joblib 1.1.0 json5 0.9.6 jsonschema 4.2.1 jupyter-client 7.1.0 jupyter-core 4.9.1 jupyter-packaging 0.11.1 jupyter-server 1.12.1 jupyterlab 3.2.4 jupyterlab-pygments 0.1.2 jupyterlab-server 2.8.2 jupyterlab-widgets 1.0.2 keras-nightly 2.5.0.dev2021032900 Keras-Preprocessing 1.1.2 kiwisolver 1.3.2 Markdown 3.3.6 MarkupSafe 2.0.1 matplotlib 3.5.0 matplotlib-inline 0.1.3 mistune 0.8.4 nbclassic 0.3.4 nbclient 0.5.9 nbconvert 6.3.0 nbformat 5.1.3 nest-asyncio 1.5.4 notebook 6.4.6 numpy 1.19.5 oauthlib 3.1.1 open3d 0.14.1 opt-einsum 3.3.0 packaging 21.3 pandas 1.3.4 pandocfilters 1.5.0 parso 0.8.3 pexpect 4.8.0 pickleshare 0.7.5 Pillow 8.4.0 pip 21.3.1 pkg_resources 0.0.0 prometheus-client 0.12.0 prompt-toolkit 3.0.23 protobuf 3.19.1 ptyprocess 0.7.0 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycparser 2.21 Pygments 2.10.0 pyparsing 3.0.6 pyrsistent 0.18.0 python-dateutil 2.8.2 pytz 2021.3 PyYAML 6.0 pyzmq 22.3.0 requests 2.26.0 requests-oauthlib 1.3.0 rsa 4.8 scikit-learn 1.0.1 scipy 1.7.3 Send2Trash 1.8.0 setuptools 59.5.0 setuptools-scm 6.3.2 six 1.15.0 sniffio 1.2.0 tensorboard 2.7.0 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.0 tensorflow 2.5.2 tensorflow-estimator 2.5.0 termcolor 1.1.0 terminado 0.12.1 testpath 0.5.0 threadpoolctl 3.0.0 tomli 1.2.2 tomlkit 0.7.2 tornado 6.1 tqdm 4.62.3 traitlets 5.1.1 typing-extensions 3.7.4.3 urllib3 1.26.7 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 1.2.1 Werkzeug 2.0.2 wheel 0.37.0 widgetsnbextension 3.5.2 wrapt 1.12.1 zipp 3.6.0

jsohn123 commented 2 years ago

@sanskar107 Any insight or help here still appreciated. Thanks!

riponcse10 commented 2 years ago

Facing the same problem. Any solution/thoughts on this?

riponcse10 commented 2 years ago

This example runs for PyTorch, not for Tensorflow. To work with Tensorflow, we need to "batchify" the data.

`test_dataset = dataset.get_split('validation') test_split = TFDataloader(dataset=test_dataset, model=model, use_cache=False)

test_loader, len_test = test_split.get_loader(batch_size=1,transform=False) print(len_test) for it in test_loader: boxes = pipeline.run_inference(it)[0] print(boxes)`