search

wdrink / Objectformer

23 stars 4 forks source link

Some unexpected results from your released checkpoints. #5

Open SunnyHaze opened 6 months ago

SunnyHaze commented 6 months ago

Hi authors!

Thanks for your outstanding work.

However, after running an evaluation script on the CASIAv1 dataset with your official checkpoint trained with CASIAv2, we found that the F1 score is 0.1918. After carefully checking the code I can not find any clues about these unexpected results.

Detailly, I implemented the F1 metrics with the third mask returned from your nn.Modules class. https://github.com/wangjk666/Objectformer/blob/a967ba91a77e79f6d64e5a02e15f4214197ceb61/ObjectFormer/models/objectformer.py#L379

Here are the visualizing results on CASIAv1:

6f20237b28ee6d78c8569f5071651ea

Do you have any ideas about it? Any help is sincerely appreciated! Thank you very much.

wscxwen commented 1 month ago

Friend, can you look at your yaml configuration file and train.txt file, I used CASIAV1 data to import some errors.