search

djghosh13 / geneval

GenEval: An object-focused framework for evaluating text-to-image alignment
https://arxiv.org/abs/2310.11513
MIT License
120 stars 6 forks source link

Issue about model_config in mmdet #6

Closed LuFan31 closed 3 months ago

LuFan31 commented 4 months ago

Hi, thank you for your great work!

I want to konw which model_config file in which version of mmdet used here when evaluating GenEval? Considering mmdet==2.27.0 needs to be installed, and 'configs/mask2former/mask2former_swin-s-p4-w7-224_lsj_8x2_50e_coco.py' is specified in code, I download the configs folder from mmdet v2.27.0, and set the configs/mask2former/mask2former_swin-s-p4-w7-224_lsj_8x2_50e_coco.py as args.model_config.

But too many weight download from download_models.sh can not match the model initialized based on the args.model_config, and all the final metrics are 0. I think this is because I don't use the correct model_config?

Best, Fan Lu

djghosh13 commented 3 months ago

Hopefully the problem is fixed by the solution proposed in #8 (the PR is also merged, so pulling the latest commit and re-installing should work as well).