Toronto3D process problems

[GeoSur]

Hello mr.Thomas.Thank you for your wonderful work!!And I have met a new question: I have modified this dataset as the same as s3dis, but when i run the triaing process, it raised "ERROR: It seems that the calibration have not reached convergence. Here are some plot to understand why:If you notice unstability, reduce the expected_N valueIf convergece is too slow, increase the expected_N value" I carefully compared my modified data with s3dis, is this because the xyz coordinate value of my input data is too large?((I have reduced the xyz coordinate value as the dataset instructions, but they are still greater than 100)

Another question: How to select an area for validation? Simply change the validation_split in s3dis?

[GeoSur]

@HuguesTHOMAS Hello, THOMAS! When I run the plot.py, it produced two mIou results like the photo. What does the line on top represent? Maybe the top is the best mIou, and the below is the mean mIou?

[HuguesTHOMAS]

So here is what it means:

• dotted line : this is the mIoU computed at each epoch, but on a small subset of the dataset (only the points that are in the input sphere that the network predicted during this validation epoch).
• normal line : this is the mIoU computed on the full dataset, so this is the real measure of the performances.

A few remarks:

• as you can see, the dotted mIoU is a lot more random, because the performances vary depending on which dataset area is being sampled at each epoch
• If you look at the validation function, every time we sample an input sphere, we project its prediction on the original point cloud. Then, every 50 epochs, we compute the mIoU on this full pointcloud.
• There are different reasons why the dotted line is on average below the normal line. The probabilities on the full cloud are averaged like votes, the subset of spheres does not have the same class distribution, changing the proportions in the confusion matrix, and thus the MIoU, etc.

[GeoSur]

@HuguesTHOMAS Thank you for your reply! Your description is really helpful! And about the output lines from the plot.py. Is the top line represented the mIou of the last epoch, and the below line represented the whole cloud too?

[HuguesTHOMAS]

Yes indeed: https://github.com/HuguesTHOMAS/KPConv-PyTorch/blob/3a774ff8d54a4d080fe65093b2299ede35d9735d/plot_convergence.py#L374-L413

line 393 = mIou of the last epoch line 408 = mIou of the last saved full cloud

[GeoSur]

@HuguesTHOMAS Haha I missed these, thank you for your reply!!