search

cure-lab / MagicDrive

[ICLR24] Official implementation of the paper “MagicDrive: Street View Generation with Diverse 3D Geometry Control”
https://gaoruiyuan.com/magicdrive/
GNU Affero General Public License v3.0
672 stars 40 forks source link

Issues related to FID calculation and BEVfusion #63

Closed zhaoyangwei123 closed 4 months ago

zhaoyangwei123 commented 4 months ago

Hello author, it seems that only partial scene images can be generated in the testing steps after training, and the generated format is also different from the Nuscenes dataset. How did you infer the entire testing set and generate a new dataset that meets the Nuscenes standard?

flymin commented 4 months ago

You can add +runner.validation_index=all to generate the full validation set.

zhaoyangwei123 commented 4 months ago

Hello, I have improved the test config according to your method, but I found that although I can infer the entire test set, the generated images still do not meet the standards of Nuscenes. In Nuscenes' samples, the images exist as a single view, but the generated images are concatenated with 6 views. At the same time, the file names used for indexing the generated images are different from those in the Nuscenes dataset samples. How can I replace the images in the old dataset with images from the new dataset?

flymin commented 4 months ago

For FID, please track progress at #32

zhaoyangwei123 commented 4 months ago

Dear author, thank you very much for your reply. However, if we only calculate FID, I think we can use the test generated ori_img and gen_img to calculate it. But compared to this, I am more concerned about how to apply it to object detection networks such as BEVFusion in the future. The current generation method seems to only be able to concatenate 6 views together. May I ask if I need to use the generated images to create a new dataset, do I need to split all the images in the test generated validation set myself and convert them into the format of nuscenes validation set?

flymin commented 4 months ago

if we only calculate FID, I think we can use the test generated ori_img and gen_img to calculate it.

You can calculate like this if you want, but this is not the standard process and we never adopt this in any case.

The current generation method seems to only be able to concatenate 6 views together.

If you look into the code, you will find out that each image is generated separately and then stitch together. You can change the code for your purpose.

zhaoyangwei123 commented 4 months ago

Okay, thank you for your reply. I will try it out.

flymin commented 4 months ago

Close for now. If you have other follow-ups, feel free to re-open.