search

cure-lab / MagicDrive

[ICLR24] Official implementation of the paper “MagicDrive: Street View Generation with Diverse 3D Geometry Control”
https://gaoruiyuan.com/magicdrive/
GNU Affero General Public License v3.0
419 stars 22 forks source link

How to generate videos in MagicDrive-t using pretrained models. #43

Closed LeningWang closed 2 weeks ago

LeningWang commented 2 weeks ago

I would like to directly obtain visualizations in MagicDrive-t using pretrained weights for rawbox_mv2.0t_0.4.3.yaml on OneDrive. Could you please provide specific execution scripts, preferably for generating frames at different time intervals? Thank you very much.

LeningWang commented 2 weeks ago

We used the following script to visualize based on the provided model files, but the generated video is completely black.

d4aad6a0f4f5f51bba2c0358d14a9d9

flymin commented 2 weeks ago

Your ann_file for testing is wrong.

We have updated the readme for the link to all pre-generated data files. Could you try again?

flymin commented 2 weeks ago

Sorry, I forgot to update some code.

It is a known issue that overflow would happen when xformers is disabled. Could you pull the latest commit and try again?

LeningWang commented 2 weeks ago

Maybe if we don't use xformers, we won't need to pull the latest commit? In other words, we only download nuscenes_mmdet3d-12Hz/nuscenes_interp_12Hz_infos_val.pkl and run python tools/test.py resume_from_checkpoint=${RUN_LOG_DIR} task_id=${ANY} runner.validation_times=4 runner.pipeline_param.init_noise=rand_all ++dataset.data.val.ann_file=${ROOT}/../data/nuscenes_mmdet3d-12Hz/nuscenes_interp_12Hz_infos_val.pkl

flymin commented 2 weeks ago

I do not understand why you refuse to take updates.

Anyway, you are free to choose to use or not to use xformers with a6b0d8d. Without it, please always enable xformers.

LeningWang commented 2 weeks ago

Thanks, my problems are well solved. Thank you, nice person, and nice work.