S3DIS Fold 5 and 6 fold differences

[jing-zhao9]

What is the difference between the S3DIS Fold 5 and 6 fold？

[drprojects]

See my reply in this previous issue: https://github.com/drprojects/superpoint_transformer/issues/73#issuecomment-1962412343

For more about S3DIS, please refer to the original paper: https://openaccess.thecvf.com/content_cvpr_2016/papers/Armeni_3D_Semantic_Parsing_CVPR_2016_paper.pdf

PS: If you ❤️ or use this project, don't forget to give it a ⭐, it means a lot to us !

[jing-zhao9]

Thank you very much for your patience！ I have another question, if I want to train 6 fold should I execute the following instructions python src/train.py experiment=semantic/s3dis_11g datamodule.fold=1 python src/train.py experiment=semantic/s3dis_11g datamodule.fold=2 python src/train.py experiment=semantic/s3dis_11g datamodule.fold=3 python src/train.py experiment=semantic/s3dis_11g datamodule.fold=4 python src/train.py experiment=semantic/s3dis_11g datamodule.fold=5 python src/train.py experiment=semantic/s3dis_11g datamodule.fold=6 After averaging the miou of each fold, 6 folds can be obtained

[drprojects]

Yes, these experiments will give you the results for each of the 6 folds. To compute the 6-fold metrics, however, you cannot take the average of the per-fold metrics, you need to merge the predictions together before computing the metrics. Said otherwise, you need to sum up the confusion matrices of each of the 6 folds.

I just pushed some code with a helper function that should do this for you. You only need to provide a dictionary pointing to the paths of each fold's pretrained model ckpt checkpoint file:

from src.utils.semantic import compute_semantic_metrics_s3dis_6fold

your_ckpt_files = {
    1: "path/to/your/fold_1/checkpoint.ckpt",
    2: "path/to/your/fold_2/checkpoint.ckpt",
    3: "path/to/your/fold_3/checkpoint.ckpt",
    4: "path/to/your/fold_4/checkpoint.ckpt",
    5: "path/to/your/fold_5/checkpoint.ckpt",
    6: "path/to/your/fold_6/checkpoint.ckpt"}

compute_semantic_metrics_s3dis_6fold(
        your_ckpt_files,
       "semantic/s3dis",
        stage='test',
        verbose=True)