search

Weifeng-Chen / control-a-video

Official Implementation of "Control-A-Video: Controllable Text-to-Video Generation with Diffusion Models"
GNU General Public License v3.0
359 stars 26 forks source link

how can i use my custom (based on sd-1.5 )model? #5

Open gonduras opened 1 year ago

gonduras commented 1 year ago

please explain thank you!

Weifeng-Chen commented 1 year ago

there's a demo in readme. have a look

G-force78 commented 1 year ago

please explain thank you!

Where did you put the model?

G-force78 commented 1 year ago

I get this error 

NameError                                 Traceback (most recent call last)

[<ipython-input-12-c66704109acf>](https://localhost:8080/#) in <cell line: 7>()
      5 state_dict_path = os.path.join(pipeline_model_path,"unet",  "/content/control-a-video/model/diffusion_pytorch_model.fp16.bin")
      6 state_dict = torch.load(state_dict_path, map_location="cpu")
----> 7 video_controlnet_pipe.unet.load_2d_state_dict(state_dict=state_dict)    # reload 2d model.

NameError: name 'video_controlnet_pipe' is not defined

from this, what is the video_controlnet_pipe ?

import os
import torch

pipeline_model_path = "/content/control-a-video/model/diffusion_pytorch_model.fp16.binl"
state_dict_path = os.path.join(pipeline_model_path,"unet",  "/content/control-a-video/model/diffusion_pytorch_model.fp16.bin")
state_dict = torch.load(state_dict_path, map_location="cpu")
video_controlnet_pipe.unet.load_2d_state_dict(state_dict=state_dict)    # reload 2d model.