search

arthurdouillard / CVPR2021_PLOP

Official code of CVPR 2021's PLOP: Learning without Forgetting for Continual Semantic Segmentation
https://arxiv.org/abs/2011.11390
MIT License
143 stars 23 forks source link

Ade20k results - which splits was used #21

Closed karlHolmquist closed 2 years ago

karlHolmquist commented 2 years ago

Hi, thank you for providing the code, I have a few questions regarding the reported results in the paper.

I noticed that both PLOP and your MIB implementation significantly out-perform the original MIB implementation on the old classes and what you only provide the Ade20k weights trained on classes 1-100.

So my questions is, which splits did you average when calculating the Ade20k results. Both the original order (a) and the random order (b)? I also had a question regarding the setting for Ade20k, is it based on the overlapped setting, the (pseudo-)disjoint setting of MIB or some other setting?

Thanks for any clarification you can give,

Best regards

arthurdouillard commented 2 years ago

  1. I've used only the original order (0 -> 149).

  2. I'm based on the overlap setting (I discuss why it's more useful in the paper), so I've rerunned MiB (with their code) in the overlap setting because they only used the disjoint setting.

karlHolmquist commented 2 years ago

Thanks, that's how I understood the paper but I wasn't completely in that aspect, I agree that the overlapped setting is the more realistic one