search

hotshotco / Hotshot-XL

✨ Hotshot-XL: State-of-the-art AI text-to-GIF model trained to work alongside Stable Diffusion XL
https://hotshot.co
Apache License 2.0
982 stars 77 forks source link

--spatial_unet_base How to use? #18

Closed wangyong860401 closed 8 months ago

wangyong860401 commented 8 months ago

(sdxl) F:\ai\Hotshot-XL>python inference.py --prompt="a bulldog in the captains chair of a spaceship, hd, high quality" --output="output2.gif" --spatial_unet_base="F:/ai/huggingface-hub/models/sdxl/models--stabilityai--stable-diffusion-xl-base-1.0/unet" --pretrained_path "F:/cache/huggingface/hub/models--hotshotco--Hotshot-XL/snapshots/41498d0b05468fa28294718e787da812b2007cd1" Traceback (most recent call last): File "F:\ai\Hotshot-XL\inference.py", line 231, in main() File "F:\ai\Hotshot-XL\inference.py", line 152, in main unet = UNet3DConditionModel.from_pretrained_spatial(args.spatial_unet_base).to(device) File "F:\ai\Hotshot-XL\hotshot_xl\models\unet.py", line 973, in from_pretrained_spatial state_dict = torch.load(model_file, map_location="cpu") File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\torch\serialization.py", line 815, in load return _legacy_load(opened_file, map_location, pickle_module, pickle_load_args) File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\torch\serialization.py", line 1033, in _legacy_load magic_number = pickle_module.load(f, pickle_load_args) _pickle.UnpicklingError: invalid load key, '\xb0'.

MichaelFan01 commented 8 months ago

I got the same problem

wangyong860401 commented 8 months ago

--spatial_unet_base How to use?

File "F:\ai\Hotshot-XL\hotshot_xl\pipelines\hotshot_xl_pipeline.py", line 840, in call noise_pred = self.unet( File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\accelerate\hooks.py", line 165, in new_forward output = old_forward(args, kwargs) File "F:\ai\Hotshot-XL\hotshot_xl\models\unet.py", line 740, in forward emb = self.time_embedding(t_emb, timestep_cond) File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\diffusers\models\embeddings.py", line 192, in forward sample = self.linear_1(sample) File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "F:\Users\GTstation\anaconda3\envs\sdxl\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 must have the same dtype

johnmullan commented 8 months ago

This is fixed in https://github.com/hotshotco/Hotshot-XL/pull/19/files