search

drprojects / superpoint_transformer

Official PyTorch implementation of Superpoint Transformer introduced in [ICCV'23] "Efficient 3D Semantic Segmentation with Superpoint Transformer" and SuperCluster introduced in [3DV'24 Oral] "Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering"
MIT License
508 stars 65 forks source link

Is it possible to have more input channels? #100

Closed narges-tk closed 2 months ago

narges-tk commented 2 months ago

Hi! Thanks for the great work! I wonder if superpoint_transformer be flexible in terms of dealing with n input channels (i.e., XYZ,RGB, and any other possible attribute)? If yes, could you please guide me what other things should be changed besides data-related things? Thanks!

drprojects commented 2 months ago

The configs system is built so that hydra dynamically parses the configs and adapts the model input size according to the chosen features. Besides, it is also possible to pick from a list of supported features to be used for superpoint partition or as input pointwise features at training time.

In invite you to have a look at the configs for the different provided datasets, some have RGB, some have lidar intensity. Besides, you want to have a close look at how the partition_hf and point_hf are used in the configs and in the project, to understand the data pipeline.

If you want to create new point-wise attributes beyond the ones we support with configs/datamodule/semantic/_features.yaml, you can. To this end, you will specify the name and dimensions of these new point attributes in this file. And if they need som careful preprocessing, you may need to build dedicated Transform operations. You will need to dive deeper into the transforms of the existing datasets to see if what you need is not already here. But everything is thoroughly commented across the project, so you should be able to read though the code.