semantic3d

[longmalongma]

I want to train my model on the semantic3d dataset, but there is no good data preprocessing method. Do you have any good suggestions?

[HuguesTHOMAS]

What do you mean by a "good" data preprocessing method?

[longmalongma]

What do you mean by a "good" data preprocessing method? I would like to use your data preprocessing approach to the Semantic3D dataset and then experiment with my own model, but I find it difficult to replace your network with mine. Do you have any good Suggestions?

[HuguesTHOMAS]

My pre-processing methods are tightly coupled with the way I coded my network model. They are not standalone easily reusable function. If you want to reproduce them, my best advice is to follow their execution and re-implement them for your own code. Or you can maybe use other libraries. You can also look into the Pytorch version of KPConv which is easier to understand.

[ShiQiu0419]

My pre-processing methods are tightly coupled with the way I coded my network model. They are not standalone easily reusable function. If you want to reproduce them, my best advice is to follow their execution and re-implement them for your own code. Or you can maybe use other libraries. You can also look into the Pytorch version of KPConv which is easier to understand.

Hi @HuguesTHOMAS ,

Do you have the results on Semantic3D Semantic-8 test set? Currently I would like to compare with SOTAs on the both test sets in order to fairly judge my approach. I have not seen KPCONV on the official semantic-8 leadboard, I would be appreciated if you can provide that.

[HuguesTHOMAS]

Hi @ShiQiu0419,

No I don't. To be fair, I only tested my method on reduced-8 as in my opinion, it is the best test set. semantic-8 is not subsampled, which means it is heavily biased towards the objects close to the scanner.

If you want to test KPConv on semantic-8, it is not that difficult, just change the test path here: https://github.com/HuguesTHOMAS/KPConv/blob/07861d8e40244c20ede5a362052e01f61dcc012a/datasets/Semantic3D.py#L138-L141

However, keep in mind that semantic-8 scenes are huge and testing them as is requires a lot of memory. If it crashes, you might want to test scenes one by one.

[ShiQiu0419]

Hi @ShiQiu0419,

No I don't. To be fair, I only tested my method on reduced-8 as in my opinion, it is the best test set. semantic-8 is not subsampled, which means it is heavily biased towards the objects close to the scanner.

If you want to test KPConv on semantic-8, it is not that difficult, just change the test path here: https://github.com/HuguesTHOMAS/KPConv/blob/07861d8e40244c20ede5a362052e01f61dcc012a/datasets/Semantic3D.py#L138-L141

However, keep in mind that semantic-8 scenes are huge and testing them as is requires a lot of memory. If it crashes, you might want to test scenes one by one.

Hi @HuguesTHOMAS ,

Many thanks for your kind suggestion. Besides, I have an empirical problem about Semantic3D: different from your approach that saves models with an interval/gap, now I select the trained model that performs better on the validation set, to generate final predictions for the online evaluation. However, this model may be a bit overfitting, and I understand it makes sense. Would you please provide me some hints about how to pick up an optimal trained model for the test set?

[HuguesTHOMAS]

In my experience, I found that choosing a model earlier in the training sometimes get you better performances, maybe because it reduces the overfitting effect. I usually choose the first checkpoints reaching convergence and not the models after.

However, this is not guaranteed and choosing the best checkpoint of trained model will not make a big difference in the final score. You might want to focus time on other choices like what is the best input radius and first_subsampling_dl to get larger gains in performances