search

QingyongHu / SQN

SQN in Tensorflow (ECCV'2022)
MIT License
101 stars 10 forks source link

How to get class weights with weak labels? #7

Open Anna0509 opened 2 years ago

Anna0509 commented 2 years ago

Hi @QingyongHu, thanks for your awesome work! You use class weighted loss while training, where class weights are calculate from full annotations, as shown in the following figure. Have you tried without using class weights? Because we can not get the class weights under the condition of weak supervision. If yes, what about the performance? image

QingyongHu commented 2 years ago

Thanks for pointing out this issue!

Yes, we incorrectly calculate the class weights from the full annotations, which actually should calculate based on the sparse annotations. Actually, the class weights under weak supervision are also accessible, just calculate based on the sparse annotated points, while ignoring the unlabeled points. Considering we randomly annotate 0.1% of the points, the class distribution should be similar.

We will re-run the experiments and get back to you.

Thanks!

Anna0509 commented 2 years ago

Hi @QingyongHu,

Thanks for your reply. Look forward to your updated results.

Anna0509 commented 2 years ago

Hi @QingyongHu,

I'd like to reproduce the results on S3DIS. My script is python main_S3DIS.py --mode test --gpu 0 --test_area 5. But I only got 56.82 mIoU after testing on the full-resolution point clouds. The best snapshot is snap-50001. The detailed result is as follows:

56.82 | 90.79 94.73 77.15 0.00 15.58 51.17 51.09 70.75 80.64 47.09 65.72 44.86 49.07

Can you please check if the provided ConfigS3DIS is correct? Or any other reasons?

Thanks!

LiXinZhana commented 1 year ago

Hi @QingyongHu,

I'd like to reproduce the results on S3DIS. My script is python main_S3DIS.py --mode test --gpu 0 --test_area 5. But I only got 56.82 mIoU after testing on the full-resolution point clouds. The best snapshot is snap-50001. The detailed result is as follows:

56.82 | 90.79 94.73 77.15 0.00 15.58 51.17 51.09 70.75 80.64 47.09 65.72 44.86 49.07

Can you please check if the provided ConfigS3DIS is correct? Or any other reasons?

Thanks! Hi, my reproduction result is similar to yours, which is very different from the result in the paper, did you solve this problem? 55.79 | 91.98 96.50 75.86 0.01 12.64 47.68 52.13 71.02 81.64 38.78 64.97 43.35 48.68

jianGao555 commented 6 months ago

Hi @QingyongHu, I'd like to reproduce the results on S3DIS. My script is python main_S3DIS.py --mode test --gpu 0 --test_area 5. But I only got 56.82 mIoU after testing on the full-resolution point clouds. The best snapshot is snap-50001. The detailed result is as follows: 56.82 | 90.79 94.73 77.15 0.00 15.58 51.17 51.09 70.75 80.64 47.09 65.72 44.86 49.07 Can you please check if the provided ConfigS3DIS is correct? Or any other reasons? Thanks! Hi, my reproduction result is similar to yours, which is very different from the result in the paper, did you solve this problem? 55.79 | 91.98 96.50 75.86 0.01 12.64 47.68 52.13 71.02 81.64 38.78 64.97 43.35 48.68

I'm getting similar results to you.