Should I retrain the pre-trained model in cross-validation on S3DIS Dataset?

[MaxMece]

when i use the provided pre-trained model, train on area 1-6 except 5 and test on area 5, i get the admirable result and mAP is about 0.6.

but when i train on area 2-6 and test on area 1, mAP is about 0.4 and then test on area 2, mAP is about 0.1.

i try many versions of mingroupsize.txt and pergroup_thres.txt, but 0.1 is still the best result.

so i have the question about the pre-trained model, maybe i shoule retrain the pre-trained model in each cross-validation?

Thanks for your response.

ps: after training 200 epochs，loss is about 300-400, and grouperr: 0.02 same: 0.01 diff: 0.001 pos: 0.004

[Xieshichen]

@ MaxMece, have you encountered the situation that when you run the valid.py there are too much nan in the results?

[Xieshichen]

@ MaxMece你能给我一份你做Scannet Evaluation的py文件不？作者给的地址已经失效了。

[longmalongma]

when i use the provided pre-trained model, train on area 1-6 except 5 and test on area 5, i get the admirable result and mAP is about 0.6.

but when i train on area 2-6 and test on area 1, mAP is about 0.4 and then test on area 2, mAP is about 0.1.

i try many versions of mingroupsize.txt and pergroup_thres.txt, but 0.1 is still the best result.

so i have the question about the pre-trained model, maybe i shoule retrain the pre-trained model in each cross-validation?

Thanks for your response.

ps: after training 200 epochs，loss is about 300-400, and grouperr: 0.02 same: 0.01 diff: 0.001 pos: 0.004

when i use the provided pre-trained model, train on area 1-6 except 5 and test on area 5, i get the admirable result and mAP is about 0.6.

but when i train on area 2-6 and test on area 1, mAP is about 0.4 and then test on area 2, mAP is about 0.1.

i try many versions of mingroupsize.txt and pergroup_thres.txt, but 0.1 is still the best result.

so i have the question about the pre-trained model, maybe i shoule retrain the pre-trained model in each cross-validation?

Thanks for your response.

ps: after training 200 epochs，loss is about 300-400, and grouperr: 0.02 same: 0.01 diff: 0.001 pos: 0.004

@maxmece can you give me a py file that you did Scannet Evaluation on? The address given by the author is invalid.

[mst136]

May I ask how to obtain the AP value about s3dis？I could run generate_results.py, but it did not have this value?

[jianuo1128]

May I ask how to obtain the AP value about s3dis？I could run generate_results.py, but it did not have this value?

@mst136 hi ,could you tell me how to prepare these txt before train? just like what mention in the this link : https://blog.csdn.net/weixin_42620141/article/details/119142860