search

SunnyHaze / IML-ViT

Official repository of paper “IML-ViT: Benchmarking Image manipulation localization by Vision Transformer”
MIT License
184 stars 23 forks source link

Poor visualization about reproducible results by Demo.ipynb and strange F1 scores #20

Closed hcztxdy closed 2 months ago

hcztxdy commented 2 months ago

The reproduced visualization is quite different from the presentation.

QQ截图20240627145524

And the F1 score is a bit strange, much higher than the essay

QQ截图20240627145507

QQ截图20240627150006

SunnyHaze commented 2 months ago

Hi! Thanks for your attention to our work.

Please refer to a closed issue to solve your visualization problem: #17. I have provided an English solution here: https://github.com/SunnyHaze/IML-ViT/issues/17#issuecomment-2076553542

From the information you have provided, it is difficult to determine the second issue. I suggest checking the generalization performance on other datasets. If the problem persists, consider using a single GPU and a single batch size to obtain prediction results accurately.

If you have further question, please feel free to reach out.

hcztxdy commented 2 months ago

Thank you very much, the first problem has been solved, and the second problem I try the method you provided.

SunnyHaze commented 2 months ago

That's great!

For second problem, I believe it is likely an overfitting issue (validated by generalization performance on other datasets. you can try using other Norm to solve this issue) or an unexpected problem during multi-GPU reduction (resolved by using a single GPU and a batch size of 1).

In our latest work IMDL-BenCo, we have addressed the multi-GPU reduction issue, ensuring accurate results even with multiple GPUs and large batch sizes. However, we are still hard working on the documentation and releasing latest features, so please stay tuned for updates.

hcztxdy commented 2 months ago

Ok, thanks for the answer.