Odd Classification Behaviour

[Man4231]

Hi Damien,

I just wanted to say thanks again for a wonderful project and congratulations on your new paper.

I have converted your project slightly to train on a custom dataset and it works relatively well. except that it leaves certain points above the actual ground surface, floating in the air (which are actually vegetation points), in the point cloud as shown by the photo below.

Do you have any ideas how to mitigate/correct this issue or where to start looking/what to change?

Thanks!

[drprojects]

Hi @Man4231, thanks for your interest in our project !

Could you please share more details of what is not working as you'd like ? I can't tell what the colors in your image represent.

• What is your screenshot representing ? An input cloud ? An output cloud ? What does the colormap represent ?
• Is the tile filtered or is this what the raw input data looks like ?
• Are you using a pretrained model that you run on your own dataset ? Are you training SPT from scratch on a new labeled dataset ?
• What do you mean by "it leaves certain points above the actual ground surface" ?
• Is the superpoint partition $P_1$ bleeding between the ground and those low vegetation points ?
• Or is it that the corresponding superpoints are incorrectly classified after training ?

PS: if you are using this project, don't forget to give it a ⭐, it matters to us !

[Man4231]

Hi Damien,

Thanks for such a quick response.

• The screenshot is an inferred cloud, the colour is simply an intensity wash (all of the points you see have been classified as ground).
• This is the raw output as it has been classified from SPT but only with the "ground" class turned on.
• I am training SPT from my own dataset so it's not pre-trained.
• If you look at the picture, you can see some points "floating" above the rest of the point cloud, where there is black background (nothing) behind them - this is not a correct classification of these isolated points - I would have expected a more homogenous output where there are not individual groups of points above the combined surface - if that makes sense.
• I'm not sure what you mean by the super point partition P1, how do I check this?
• I have not looked into the specific super points - only looked at the lowest level of the NAG/individual points. Could you point me to where I could check this?

Thanks!

[drprojects]

Hi @Man4231

I have a few more questions to be 100% sure I understand.

The screenshot is an inferred cloud, the colour is simply an intensity wash (all of the points you see have been classified as ground).

• So, the color we see corresponds to your predictions, with the colormap attributed to the "ground" class in your dataset ? Or is it the LiDAR intensity of your input cloud (ie something like NAG[0].intensity) ? This is the raw output as it has been classified from SPT but only with the "ground" class turned on.
• As of now, the raw output of SPT is actually a classification on $P_1$ (ie the first level superpoint partition, what you find in NAG[1]). This means obtaining the corresponding classification on $P_0$ (ie the points, what you find in NAG[0]) requires a little bit of reindexing work (this is a feature I will soon release, by the way). Have you coded that yourself, using the indices in NAG[0].super_index ? Otherwise, I am not sure how you could show this illustration of predicted classification on the $P_0$ level. Are you maybe showing NAG[0].y in this illustration ?

[drprojects]

Without a reply from you, may I close this issue ?

[Man4231]

Hi Damien,

Sorry for the delay.

• The points in that picture are actually all the same class, they have just had an "intensity wash" applied to them ie. to see more features, it is only a visual aide. But they are all the same predicted class which is ground.
• I have passed in an source index from our data source as a feature, which is then used to map the results back to the original location in our data.
• It is definitely a prediction outcome not the original classification data.
• I've put a similar photo without the intensity wash below so you can see the ground class with "floating" points.
• I've also added the same photo with all the other classes as well (floating points are taken from the vegetation class).
• For good measure I've also included the same photo with all classes on with the intensity wash.

Any ideas how we can fix this issue?

Thanks Man4231

[drprojects]

So, from your answer I am not 100% certain, but I assume you managed to produce full-resolution outputs from per-superpoint predictions using NAG[0].super_index.

You should visualize the superpoint partition $P_1$, to see if the points whose classification you find unsatisfying are isolated in their own segments or if ground segments are bleeding into vegetation a bit. To this end, we provide visualization tools in the repo that could help you with that, I invite you to have a look at the documentation and notebooks to get started.

I do not know what your downstream task is, but it seems to me that the model is still doing a pretty good job at separating ground and vegetation. If these "flying" ground points are a big problem for you, you can:

• design a simple post-processing on points classified as ground to remove them
• adjust the parameters of the partition and the graph construction to isolate these points in their own superpoints. There is no secret go-to recipe for that, you will need to adjust parameters to your dataset's properties. You can find more details on the partition hyperparameters in the documentation and in the code (which is fairly commented). Also, see my answer to this issue for some pointers https://github.com/drprojects/superpoint_transformer/issues/36

PS: if you are using this project, don't forget to give it a ⭐, it matters to us !