search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
19.49k stars 2.48k forks source link

BEIT2 ADE20 fine-tuning results do not match the accuracy reported in the github #1110

Open jsrdcht opened 1 year ago

jsrdcht commented 1 year ago

Describe the bug Model I am using (UniLM, MiniLM, LayoutLM ...): BEIT 2

The problem arises when using:

A clear and concise description of what the bug is.

I downloaded beitv2_base_patch16_224_pt1k on Github, downloaded the code for semantic segmentation, and ensured consistent environment through Docker. However, I only obtained 52.63 in fine-tuning. Even when I tried multiple experiments with my own environment, the highest score I got was only 52.8. However, Github shows that there should be 53.1 with direct fine-tuning.

pengzhiliang commented 1 year ago

Hello, thanks for the attention! I am sorry to hear that.

I have uploaded the training log here: 20220531_150806.log

Please feel free to contact if you need further information.

caddyless commented 1 year ago

I met the same issue exactly. I finetuned the same checkpoint strictly following the official instructions, but only got 52.26 mIoU finally, which is far behind the reported one 53.1. I have checked the provided log file and found that all configurations are consistent except the pytorch version (provided 1.6.0, reproduced 1.7.1). I would appreciate it if anyone could give me some insights.