search

filaPro / oneformer3d

[CVPR2024] OneFormer3D: One Transformer for Unified Point Cloud Segmentation
Other
348 stars 32 forks source link

visualize the predicted result #72

Closed TangKexin4646 closed 2 months ago

TangKexin4646 commented 3 months ago

How can I visualize the predicted result for the instance segmentation task?

oneformer3d-contributor commented 3 months ago

Please check #57.

TangKexin4646 commented 3 months ago

Thank you for the reply.

And I changed the oneformer3d.py according to https://github.com/oneformer3d/oneformer3d/issues/57 I also write the predicted code according to https://github.com/oneformer3d/oneformer3d/issues/67

the code is as follows:

from mmengine.config import Config from mmengine.registry import Registry, build_functions import numpy as np model_registry = Registry('model') model_config_path = '/media/tang/shared/oneformer3d-main/configs/oneformer3d_1xb2_s3dis-area-5.py' model_config = Config.fromfile(model_config_path) checkpoint = '/media/tang/shared/oneformer3d-main/work_dirs/oneformer3d_1xb2_s3dis-area-5/epoth_512.pth' pcd_path = '/media/tang/shared/oneformer3d-main/data/s3dis/points/Area_1_conferenceRoom_1.bin' pcd = np.fromfile(pcd_path, dtype=np.float32).reshape(-1, 6) model = build_functions.build_model_from_cfg(model_config , model_registry) model.load_checkpoint(checkpoint) result = model.predict(dict(points=pcd))

but there is an error: File "/media/tang/shared/oneformer3d-main/s3dis_predict.py", line 20, in model = build_functions.build_model_from_cfg(model_config , model_registry) File "/home/tang/anaconda3/envs/mmdetection3d/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg return build_from_cfg(cfg, registry, default_args) File "/home/tang/anaconda3/envs/mmdetection3d/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 72, in build_from_cfg raise KeyError( KeyError: 'cfg or default_args must contain the key "type", but got Config (path: /media/tang/shared/oneformer3d-main/configs/oneformer3d_1xb2_s3dis-area-5.py): {\'default_scope\': \'mmdet3d\', \'defa.....)

Could you please help me figure out the problem?

oneformer3d-contributor commented 3 months ago

Looks like in model_config should be Config.fromfile(model_config_path)['model'].

Also it should be possible to build runner from config like here and then access the model as runner.model.

TangKexin4646 commented 3 months ago

model_config should be Config.fromfile(model_config_path)['model'] Yes, it works, but there is a new error.

Traceback (most recent call last): File "/media/tang/shared/oneformer3d-main/s3dis_predict.py", line 21, in model = build_functions.build_model_from_cfg(model_config , model_registry) File "/home/tang/anaconda3/envs/mmdetection3d/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg return build_from_cfg(cfg, registry, default_args) File "/home/tang/anaconda3/envs/mmdetection3d/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 100, in build_from_cfg raise KeyError( KeyError: 'S3DISOneFormer3D is not in the main::model registry. Please check whether the value of S3DISOneFormer3D is correct or it was registered as expected. More details can be found at

  1. Also it should be possible to build runner from config like here and then access the model as runner.model.

I want to predict a single point cloud data like https://github.com/oneformer3d/oneformer3d/issues/67, but the runner seems to expect lots of data.

zeyu659 commented 3 months ago

@TangKexin4646 Hi, Previously the author said that pred is inconsistent with gt labels, so I have a problem with the visualization code, unfortunately, I can't help you to solve the problem, because I haven't found a corresponding method myself.

oneformer3d-contributor commented 3 months ago

To fix import error you need to use custom_imports key from our config file. To do it please call something like import_modules_from_strings(**cfg_dict['custom_imports']) like here.