Open xinli2008 opened 5 months ago
Please share the code as code and not as a screenshot and make it fully reproducible.
from diffusers import DiffusionPipeline from safetensors.torch import load_file import torch from PIL import Image
pipe = DiffusionPipeline.from_pretrained("/mnt/asian-t2i/pretrained_models/RealVisXL_V3.0", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
pipe.to("cuda:2")
pipe.load_textual_inversion("/mnt/asian-t2i/output/textual-inversion/textual_inversion-sdxl-data1k6-realism/checkpoint-1000/model.safetensors", token="
We don't know where the following come from:
The checkpoint-1000/model.safetensors is the generated model after i train the textual inversion sdxl model from here
We don't have the bandwidth to do a training run to reproduce the issue here. Please share the checkpoint too.
And in your screenshot, you're supplying a token and and in the code, you didn't supply token. Please make sure the code is exactly what you're looking for.
I trained a textual inversion model on SDXL pretrained model ( RealVisXL3.0 ), and then when I want to inference, I use this textual inversion model as a positive prompt to strengthen some aspect of the model.
Try providing a code snippet for this too as this will help us have all the info we need.
First, the generated textual inversion safetensors is local and how can i share it with you?
Second, i use the following code to load_textual_inversion:
pipe = DiffusionPipeline.from_pretrained("/mnt/asian-t2i/pretrained_models/RealVisXL_V3.0", torch_dtype=torch.float16, use_safetensors=True, variant="fp16") pipe.to("cuda:2") pipe.load_textual_inversion("/mnt/asian-t2i/output/textual-inversion/textual_inversion-sdxl-data1k6-realism/checkpoint-1000/model.safetensors", token="abc-person-realism", text_encoder=pipe.text_encoder, tokenizer=pipe.tokenizer)
First, the generated textual inversion safetensors is local and how can i share it with you?
You can host it on the Hugging Face Hub temporarily and share with us the link.
thank you very much and i will try that
Cc: @yiyixuxu could you give this a look?
And i have another question, i read the readme from here.
The example embedding strtcture is like this:
It is so different from the generated file, i am wondering why?
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
@xinli2008 Could you share the command you used to train your textual inversion model please?
Also I think you've uploaded a full text model rather than the inversion embedding here. Which is why I think you're seeing this error.
I'm facing the same problem. The same LoRA works fine with “load_lora_weights” but fails with “load_textual_inversion”. *LoRA uses local files
successful
pipe.load_lora_weights(".", weight_name="./lora/hoge.safetensors")
fail
pipe.load_textual_inversion("./lora/hoge.safetensors", token="hoge", local_files_only=True)
error
Please verify that the loaded state dictionary of the textual embedding either only has a single key or includes the
string_to_param
input key.
@gulatis A LoRA isn't the same thing as a textual inversion. So it makes sense that the loading would fail.
@DN6 I see. So it was for embedding, thank you.
Describe the bug
I trained a textual inversion model on SDXL pretrained model ( RealVisXL3.0 ), and then when I want to inference, I use this textual inversion model as a positive prompt to strengthen some aspect of the model. But when I try to load the RealVisXL3.0 with
from_pretrained()
first and load the textual-inversion model withload_textual_inversion(),
I get an error. I looked at the relevant information, but there was no good workaround.Reproduction
Mycode:
Error:
ValueError: Loaded state dictionary is incorrect: {'text_model.embeddings.position_embedding.weight'
Please verify that the loaded state dictionary of the textual embedding either only has a single key or includes the
string_to_param
input key.Logs
System Info
diffusers version: Name: diffusers Version: 0.27.0.dev0
Who can help?
@sayakpaul @yiyixuxu @DN6