Open mostafa501 opened 8 months ago
Yes, some script like this should work. Btw, TD3D uses mmdet3d version 1.0, and for OneFormer3D it is 1.1. And there were a lot of changes between these two versions.
I also want to train my own data, my data only has two semantic categories, how can I change the code in the configuration file (https://github.com/filaPro/oneformer3d/blob/main/configs/oneformer3d_1xb2_s3dis-area-5.py)?
-- (L200) stuff_cls=[0, 1, 2, 3, 4, 5, 6, 12], thing_cls=[7, 8, 9, 10, 11])) What do these two lines of code mean? I only have two categories. How can I change them?
These stuff_cls
and thing_cls
are only used for panoptic evaluation. You can simply delete panoptic segmentation call from evaluation. Or call your background as 0 class, and your two instance classes as 1 and 2.
@filaPro Thank you for the reply, To run the test demo code, I tried first to install required libararies and run it. But I got this error:
egmentation/oneformer3d-main$ /home/navlab/anaconda3/envs/oneformer3d1/bin/python /media/navlab/GNSS/work/instance_segmentation/oneformer3d-main/demo/pc_seg_demo.py
Traceback (most recent call last):
File "/media/navlab/GNSS/work/instance_segmentation/oneformer3d-main/demo/pc_seg_demo.py", line 28, in <module>
model = init_model(args.config, args.checkpoint, device=args.device)
File "/media/navlab/GNSS/work/instance_segmentation/oneformer3d-main/mmdetection3d/mmdet3d/apis/inference.py", line 59, in init_model
config = Config.fromfile(config)
File "/home/navlab/anaconda3/envs/oneformer3d1/lib/python3.8/site-packages/mmengine/config/config.py", line 462, in fromfile
import_modules_from_strings(**cfg_dict['custom_imports'])
File "/home/navlab/anaconda3/envs/oneformer3d1/lib/python3.8/site-packages/mmengine/utils/misc.py", line 77, in import_modules_from_strings
imported_tmp = import_module(imp)
File "/home/navlab/anaconda3/envs/oneformer3d1/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 843, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/media/navlab/GNSS/work/instance_segmentation/oneformer3d-main/oneformer3d/oneformer3d.py", line 4, in <module>
from torch_scatter import scatter_mean
File "/home/navlab/.local/lib/python3.8/site-packages/torch_scatter/__init__.py", line 16, in <module>
torch.ops.load_library(spec.origin)
File "/home/navlab/anaconda3/envs/oneformer3d1/lib/python3.8/site-packages/torch/_ops.py", line 852, in load_library
ctypes.CDLL(path)
File "/home/navlab/anaconda3/envs/oneformer3d1/lib/python3.8/ctypes/__init__.py", line 373, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libcudart.so.11.0: cannot open shared object file: No such file or directory
The error seems to be related to CUDA version, so I checked my GPU CUDA version, can you tell me if I can install this model or what shall I do to install formerone3d network model (where I got 2 versions of CUDA) ? Thank you.
(oneformer3d1) navlab@navlab-ProLiant-DL380-Gen10:/media/navlab/GNSS/work/instance_segmentation/oneformer3d-main/mmdetection3d$ nvidia-smi
Thu Dec 28 21:04:29 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.223.02 Driver Version: 470.223.02 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla V100-PCIE... Off | 00000000:37:00.0 Off | 0 |
| N/A 48C P0 29W / 250W | 4MiB / 32510MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2215 G /usr/lib/xorg/Xorg 4MiB |
+-----------------------------------------------------------------------------+
(oneformer3d1) navlab@navlab-ProLiant-DL380-Gen10:/media/navlab/GNSS/work/instance_segmentation/oneformer3d-main/mmdetection3d$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
nvcc --version
should show exactly 11.6 version of cuda. This is because you follow our Dockerfile, we download mmcv, torch-scatter and spconv only for cuda 11.6.
@filaPro thank you so much for this amazing work. I tried to play around with a solution similar to what @mostafa501 posted above, which is what mmdet3d suggests, but I encountered some issues related to some internal mmdet structure.
I believe the code is trying to find the ground-truth for the provided pointcloud. I've seen that you encountered and solved a similar issue here, and similar errors are happening to me as well.
Could you please provide a working example that runs the method on a custom input pointcloud without any labels available? I think it would be a great addition to the repository, as e.g. Mask3D proposes, so everyone will be able to play with the method and see how it works on their own data.
Unfortunately don't have much time now as switching to a new job in a new country.
Don't quite understand why smth like demo.ipynb shouldn't work. For sure it doesn't need annotations. In our test pileline for S3DIS dataset we return only point. You can use the same pipeline, just remove the LoadAnnotations3D
here. For ScanNet it is much more difficult as we use superpoint clustering before running the model.
These
stuff_cls
andthing_cls
are only used for panoptic evaluation. You can simply delete panoptic segmentation call from evaluation. Or call your background as 0 class, and your two instance classes as 1 and 2.
I have deleted the panoptic segmentation call by commenting line no. 137 to 140 and ret_pan
in line 173 in this file, but I am getting error while evaluation as
File "/workspace/oneformer3d/oneformer3d.py", line 952, in pred_sem
seg_map = mask_pred.argmax(0)
IndexError: argmax(): Expected reduction dim 0 to have non-zero size.
These and are only used for panoptic evaluation. You can simply delete panoptic segmentation call from evaluation. Or call your background as 0 class, and your two instance classes as 1 and 2.
stuff_cls``thing_cls
I have deleted the panoptic segmentation call by commenting line no. 137 to 140 and in line 173 in this file, but I am getting error while evaluation as
ret_pan
File "/workspace/oneformer3d/oneformer3d.py", line 952, in pred_sem seg_map = mask_pred.argmax(0) IndexError: argmax(): Expected reduction dim 0 to have non-zero size.
Have you solved it now?
@Lizhinwafu Yes, I was able to solve it!
mmdetection3d/tools/dataset_converters/s3dis_data_utils.py
based on the your dataset. Modification are to be done at line no. 28, 180, and 181.mmdetection3d/tools/dataset_converters/update_infos_to_v2.py
at line 538.METAINFO
in oneformer3d/s3dis_dataset.py
by changing the class names and countconfigs/oneformer3d_1xb4_scannet.py
by changing the num_instance_classes
, num_semantic_classes
, class_names
, etc. There are many changes required in this file (doing this is quite intuitive though)With this, I was able to train on my custom dataset!
Many thanks.
------------------ Original ------------------ From: Prathamesh Dinkar @.> Date: Thu,Jul 25,2024 5:07 AM To: oneformer3d/oneformer3d @.> Cc: Lizhi Jiang @.>, Mention @.> Subject: Re: [oneformer3d/oneformer3d] How to apply the network on my owndataset using test code (Issue #9)
@Lizhinwafu Yes, I was able to solve it!
Modify the classes ids in mmdetection3d/tools/dataset_converters/s3dis_data_utils.py based on the your dataset. Modification are to be done at line no. 28, 180, and 181.
Modify the class names in mmdetection3d/tools/dataset_converters/update_infos_to_v2.py at line 538.
Modify the METAINFO in oneformer3d/s3dis_dataset.py by changing the class names and count
Modify configs/oneformer3d_1xb4_scannet.py by changing the num_instance_classes, num_semantic_classes, class_names, etc. There are many changes required in this file (doing this is quite intuitive though)
With this, I was able to train on my custom dataset!
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>
Hi, thank you so much for publishing this work, we want to use this network for better accuracy. So we ask how to test our own collected data as we did before in TD3D network. This is the demo test used in TD3D, is it available to use a snippet like this or not:
Thank you so much for the continuous help.