search

cure-lab / MagicDrive

[ICLR24] Official implementation of the paper “MagicDrive: Street View Generation with Diverse 3D Geometry Control”
https://gaoruiyuan.com/magicdrive/
GNU Affero General Public License v3.0
628 stars 37 forks source link

How to use the pretrained weight "SDv1.5mv-rawbox_2023-09-07_18-39_224x400" to launch another train? #56

Closed kila9112 closed 3 months ago

kila9112 commented 3 months ago

Thank you for your work! I want to train my data with the pretrained weight "SDv1.5mv-rawbox_2023-09-07_18-39_224x400",but lose "pytorch_model.bin". And if i train the model with my own "pytorch_model.bin", it raised error "pytorchstreamreader failed reading zip archive: failed finding central direc". How can i train with a pretrained weight? Thank you very much!

flymin commented 3 months ago

Please refer to the video branch, which loads image generation for video generation training.

For your problem, it seems like a pytorch env problem or error in the downloaded file. Please make sure you can run the inference before you start.

kila9112 commented 3 months ago

Thank you for your suggention. I find another resolution of 272×736 is also supported, how can i change the code?

flymin commented 3 months ago

Yes, we do not include the implementation here. You can change the resolution through config, but you have to change the code for BEV map encoder, to have the same embedding size as the image latent (similar logic as ControlNet). Other parts should be fine.