search

zbf1991 / SCL

37 stars 7 forks source link

why did the model of SCL_PFENET trained by myself fail to achievel the performance of the paper? miou just 60.283 #4

Closed TizzyTTT closed 3 years ago

TizzyTTT commented 3 years ago

I download the paper code and trained the model (SCL PFENET ) for 200 epochs without any change, pytorch version is 1.6.0. But the best Miou is just 60.28

Long before , I also trained PFENet by myself. And I got the similar results mIoU 61.4~61.6

TizzyTTT commented 3 years ago

backbone: resnet50 dataset: pasical 5i fold0

Saralyliu commented 3 years ago

we kept with pytorch vision as PFENET, trained the model SCL and only got a result 61.4. The PFENET, we got 60.89, how your reproduced the PFENet, did you try other splits? I can't got the results.

TizzyTTT commented 3 years ago

we kept with pytorch vision as PFENET, trained the model SCL and only got a result 61.4. The PFENET, we got 60.89, how your reproduced the PFENet, did you try other splits? I can't got the results.

I think the framework of PEFNet has no problems. I' ve also tried other splits and there is not much difference with the paper. How about changing the seed number or make the batch size 6 ? or change a GPU.

TizzyTTT commented 3 years ago

we kept with pytorch vision as PFENET, trained the model SCL and only got a result 61.4. The PFENET, we got 60.89, how your reproduced the PFENet, did you try other splits? I can't got the results.

How about updating the version of pytorch ? A higher version maybe plays a role.

Saralyliu commented 3 years ago

we kept with pytorch vision as PFENET, trained the model SCL and only got a result 61.4. The PFENET, we got 60.89, how your reproduced the PFENet, did you try other splits? I can't got the results.

I think the framework of PEFNet has no problems. I' ve also tried other splits and there is not much difference with the paper. How about changing the seed number or make the batch size 6 ? or change a GPU.

The author of PFENET suggested 1.4.0 pythorch and a GPu, we followed the setting, without change the code and cannot get the
result. what are your setting details.

zbf1991 commented 3 years ago

I will check again for the code to see if I have uploaded something incorrectly. Sorry for that

zbf1991 commented 3 years ago

I am re-running my code and I will reply to you as soon as possible.

zbf1991 commented 3 years ago

Hello, I have re-run my code, using one 2080Ti and the pytorch version is 1.4. before 50 epochs (46 epoch), it has reached 62.5% mIoU. I am also testing 1080Ti and pytorch 1.3. If you can change "train.py" line 271, "epoch % 2" to "epoch%1" and re-run this code (I have not done this). I will keep this issue open and for any results please tell me.

TizzyTTT commented 3 years ago

Hello, I have re-run my code, using one 2080Ti and the pytorch version is 1.4. before 50 epochs (46 epoch), it has reached 62.5% mIoU. I am also testing 1080Ti and pytorch 1.3. If you can change "train.py" line 271, "epoch % 2" to "epoch%1" and re-run this code (I have not done this). I will keep this issue open and for any results please tell me.

Thank you for replying. I' ll check it again.

zbf1991 commented 3 years ago

I have re-uploaded my 1-shot file, which is the current file I just re-tun. Although there is no difference. You can download this project and re-run it, Besides, 1080Ti with pytorch1.4 is also okay (epoch 37 can reach 63.0). 2080ti with pytorch1.3 can also work. And it seems that pytorch1.3 is more stable. If you want to see the log file I can share it with you.

TizzyTTT commented 3 years ago

I have re-uploaded my 1-shot file, which is the current file I just re-tun. Although there is no difference. You can download this project and re-run it, Besides, 1080Ti with pytorch1.4 is also okay (epoch 37 can reach 63.0). 2080ti with pytorch1.3 can also work. And it seems that pytorch1.3 is more stable. If you want to see the log file I can share it with you.

Feel sorry, I change the batch size from 4 to 6 last time. This time I keep all the setting as yours and the performance reaches the 62.85 in epoch 46 (resnet50 pascal5i fold0 miou 0.6285). it is very close

zbf1991 commented 3 years ago

I have re-uploaded my 1-shot file, which is the current file I just re-tun. Although there is no difference. You can download this project and re-run it, Besides, 1080Ti with pytorch1.4 is also okay (epoch 37 can reach 63.0). 2080ti with pytorch1.3 can also work. And it seems that pytorch1.3 is more stable. If you want to see the log file I can share it with you.

Feel sorry, I change the batch size from 4 to 6 last time. This time I keep all the setting as yours and the performance reaches the 62.85 in epoch 46 (resnet50 pascal5i fold0 miou 0.6285). it is very close.

Thanks for your reply. Note that the val_pair in train.py is 2000, you can change it to 5000 to get the final performance, as done in the original PFENET. If there is no further problem, I will close this issue.