search

drprojects / superpoint_transformer

Official PyTorch implementation of Superpoint Transformer introduced in [ICCV'23] "Efficient 3D Semantic Segmentation with Superpoint Transformer" and SuperCluster introduced in [3DV'24 Oral] "Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering"
MIT License
560 stars 72 forks source link

Inference with pretrain model #8

Closed iacopo97 closed 1 year ago

iacopo97 commented 1 year ago

Good morning, thank you for sharing this repository and for your magnificent work. I would like to know, if it is possible to use one of your model pretrained, for example in Kitti3D, to do inference about a new point cloud. In my case i would like to do a 3D semantic segmentation on an acquired pointcloud. Thank you very much!

drprojects commented 1 year ago

Hello, thanks for your interest in this project !

You can definitely use the pretrained models on your own data. To this end, you will need to create your dataset. I refer you to the instructions for creating your own dataset.

Besides, if the dataset you are using is open, we would gladly welcome pull requests from integrating new datasets in the project :wink:

iacopo97 commented 1 year ago

Thank you very much for your kind answer, I will work on that. Unfortunately, it is not an open dataset.🤣

drprojects commented 1 year ago

Too bad, integrating a new large-scale dataset would be great ! We will try to add more open datasets to this project in the future. Good luck with yours in the meantime ! :blush:

iacopo97 commented 1 year ago

Thank you very much, good luck with your work too, do you think that the network that you proposed in this repo could be useful in the 3D lane detection task?

drprojects commented 1 year ago

I am not extensively familiar with 3D Lane Detection, but from what I gather, this task seems heavily guided by radiometric information. Said otherwise, if you wanted to tackle this task using a point cloud based method like SPT, you would need to be sure your points are properly colorized: no 3D-2D misalignments, well-calibrated colors, etc.

Again, I do not hold much expertise for this specific task, but I would assume the 3D geometric information is not as important for lane segmentation as it is for 3D semantic segmentation datasets such as S3DIS and KITTI-360. If so, the geometric information carried by points would not be so useful for lane segmentation and you might be better-off segmenting lanes directly in your images and projecting the result on your 3D points, if you have access to the camera model and pose.

By the way, if you happen to be interested in methods for jointly extracting features from point clouds and multi-view images, you can check our CVPR'22 work: https://github.com/drprojects/DeepViewAgg.

Good luck !

iacopo97 commented 1 year ago

Thank you very much for your advice, it has been extremely useful, I will definitely check the repository that you have shared.