search

mczhuge / ICON

(TPAMI2022) Salient Object Detection via Integrity Learning.
MIT License
180 stars 14 forks source link

Mismatched Evaluation Results #10

Closed gbliao closed 1 year ago

gbliao commented 1 year ago

Hi, I have tested with the ResNet50 weights and dataset provided by GitHub and have not modified the code. However, I got the following results, which are inconsistent with the paper's results.

Sm:0.812 maxFm:0.743 MAE:0.062 adpFm:0.735 meanFm:0.726 || Data:[_ICON_git/DUT-OMRON]] Sm:0.854 maxFm:0.824 MAE:0.047 adpFm:0.794 meanFm:0.804 || Data:[_ICON_git/DUTS]] Sm:0.886 maxFm:0.889 MAE:0.045 adpFm:0.872 meanFm:0.865 || Data:[_ICON_git/HKU-IS]] Sm:0.845 maxFm:0.833 MAE:0.072 adpFm:0.803 meanFm:0.814 || Data:[_ICON_git/PASCAL-S]] Sm:0.903 maxFm:0.911 MAE:0.047 adpFm:0.889 meanFm:0.890 || Data:[_ICON_git/ECSSD]]

gbliao commented 1 year ago

Can you help me to solve it? Thank you very much!

mczhuge commented 1 year ago

Could you double-check the line: https://github.com/mczhuge/ICON/blob/main/main/train.py#119.

There are another one also faced this issue before, but solved by solving the environment.

And for quick contacting, you can add my WeChat: tjpxiaoming.

mczhuge commented 1 year ago

Have you solved yet? @gbliao

gbliao commented 1 year ago

Sorry to reply so late.

I actually load the pre-training weights provided in GitHub to test, and this process does not involve training-related settings.

mczhuge commented 1 year ago

It is so weird. Could you also try Swin, PVT as the encoder?

gbliao commented 1 year ago

I have not tested the results with Swin as the encoder.

Do you retest the results of ResNet as the encoder properly?

mczhuge commented 1 year ago

Thanks for your carefulness.

After I checked, I found this is a bug originated from https://github.com/mczhuge/ICON/issues/4.

I will re-upload the checkpoints in the several weeks.