search

cure-lab / MagicDrive

[ICLR24] Official implementation of the paper “MagicDrive: Street View Generation with Diverse 3D Geometry Control”
https://gaoruiyuan.com/magicdrive/
GNU Affero General Public License v3.0
419 stars 22 forks source link

The released weight fails to reproduce performance as reported #22

Closed ppeterpp closed 3 months ago

ppeterpp commented 3 months ago

Hi! Images generated via weights released led to worse reported numbers compared to those in the paper, may I ask how many epochs of training required for reproduction, and the epochs you have trained for your released weights, thanks.

flymin commented 3 months ago

The released pre-trained weight is exactly the one we used for evaluation in our paper. And the config is used to train this weight.

ppeterpp commented 3 months ago

we modified your test.py to gen full val set and bicubic upsample followed by pad top, as described in another issue. Then we swap the gen val set to nus and do cvt & bevfusion validation without modifying the downstream codes. could there be any thing wrong in our practice, please shed some light!

flymin commented 3 months ago

As your descriptions seem to match our paper, they are correct. Please look into more details to discover any possible issues in your implementation. Otherwise, please wait patiently for our future updates.

If this is the answer you want, there is no special trick in our evaluation (e.g., we used all scenes from the val dataset and all boxes). Sorry, I cannot help you with such a general question. I will close this issue.

If you have any specific problems, please open other issues and post them there, just like others did.