search

kaixin96 / PANet

Code for our ICCV 2019 paper PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment
315 stars 64 forks source link

The accuracy is lower #22

Closed luckybeanss closed 4 years ago

luckybeanss commented 4 years ago

Thanks for sharing your code. It is a really great work! But the accuracy rate I got based on this code is lower than that reported in the paper. Can you share the hyperparameter settings on the VOC2012 dataset？

kaixin96 commented 4 years ago

Hi @luckybeanss, hyper-parameters are either mentioned in the paper or included in config.py. May I know which hyper-parameters are missing? Thank you.

By the way, if the performance gap is less than 0.5 mIoU, it could be due to randomness.

Best regards, Kaixin

luckybeanss commented 4 years ago

Thanks for your reply. According to the parameters in the code and paper, the 1-shot accuracy I got on the PASCAl-5i dataset is 41%. So I want to know which parts need to pay attention。

kaixin96 commented 4 years ago

@luckybeanss you mean the mIoU averaged across 4 splits is 41% or just the first split?

qiulesun commented 4 years ago

@kaixin96 How can I run your code with two GPU cards when the batch_size is set to 4 for fast training ? Could you provide the corresponding modifications and run command ？

kaixin96 commented 4 years ago

Hi @qiulesun, as your question is not relevant to this issue, can you open a new one? It will be easier for other people to search similar questions. Thank you.

tymatfd commented 4 years ago

I got lower accuracy too. About 41% for 1-way 1-shot trained with default settings in the code, 47% for 1-way 5-shot using trained VOC_1way5shot_set3 model, is there any hyper parameters needed to be modified for default setting ?

kaixin96 commented 4 years ago

Hi @tymatfd, the performance reported in the paper for "1-way 5-shot split-4" is 46.5, which is close to your result. Note that set_3 actually refers to split-4 in the paper. Hope this clarifies your question.

Thank you.

tymatfd commented 4 years ago

Ah, Thank you very much @kaixin96 . So the accuracy of model trained with default settings is actually calculated on split-1 ? which is reported to be 42.3

kaixin96 commented 4 years ago

You are right. The configuration for the split is in https://github.com/kaixin96/PANet/blob/1695bf37c2f7585323e09cfcfb3bef8d4fff1b86/config.py#L37

Thank you.

kaixin96 commented 4 years ago

I’m closing this issue because it has been inactive for a while. Feel free to reopen if you have questions. Thank you.