Doubiiu
commented
5 months ago Hi Theis is caused by loading the DynamiCrafter256 model checkpoint into the DynamiCrafter1024 model. (because of the bug in checkpoint-downloading script i2v_test.py and now I have fixed it.). You can delete the ckpt file and redownload it manually from DynamiCrafter_1024 or download using the updated code.
(S:\DynamiCrafter) S:\DynamiCrafter> python gradio_app.py --res 1024 A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton' AE working on z of shape (1, 4, 32, 32) = 4096 dimensions. Traceback (most recent call last): File "S:\DynamiCrafter\scripts\evaluation\funcs.py", line 106, in load_checkpoint model.load_state_dict(state_dict, strict=full_strict) File "S:\DynamiCrafter\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for LatentVisualDiffusion: Missing key(s) in state_dict: "scale_arr", "model.diffusion_model.fps_embedding.0.weight", "model.diffusion_model.fps_embedding.0.bias", "model.diffusion_model.fps_embedding.2.weight", "model.diffusion_model.fps_embedding.2.bias". Unexpected key(s) in state_dict: "model.diffusion_model.framestride_embed.0.weight", "model.diffusion_model.framestride_embed.0.bias", "model.diffusion_model.framestride_embed.2.weight", "model.diffusion_model.framestride_embed.2.bias", "model.diffusion_model.input_blocks.1.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.2.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.4.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.5.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.7.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.8.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.middle_block.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.3.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.4.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.5.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.6.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.7.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.8.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.10.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.11.1.transformer_blocks.0.attn2.alpha".
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "gradio_app.py", line 170, in
dynamicrafter_iface = dynamicrafter_demo(result_dir, args.res)
File "gradio_app.py", line 47, in dynamicrafter_demo
image2video = Image2Video(result_dir, resolution=resolution)
File "S:\DynamiCrafter\scripts\gradio\i2v_test.py", line 31, in init
model = load_model_checkpoint(model, ckpt_path)
File "S:\DynamiCrafter\scripts\evaluation\funcs.py", line 127, in load_model_checkpoint
load_checkpoint(model, ckpt, full_strict=True)
File "S:\DynamiCrafter\scripts\evaluation\funcs.py", line 118, in load_checkpoint
model.load_state_dict(new_pl_sd, strict=full_strict)
File "S:\DynamiCrafter\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentVisualDiffusion:
Missing key(s) in state_dict: "scale_arr".
Unexpected key(s) in state_dict: "model.diffusion_model.input_blocks.1.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.2.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.4.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.5.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.7.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.input_blocks.8.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.middle_block.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.3.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.4.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.5.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.6.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.7.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.8.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.10.1.transformer_blocks.0.attn2.alpha", "model.diffusion_model.output_blocks.11.1.transformer_blocks.0.attn2.alpha".
(S:\DynamiCrafter) S:\DynamiCrafter>