search

leejet / stable-diffusion.cpp

Stable Diffusion and Flux in pure C/C++
MIT License
3.5k stars 300 forks source link

add support for load textual inversion embeddings #335

Open zhaojinzhou opened 3 months ago

zhaojinzhou commented 3 months ago

hope to support embeddings format end with .pt

daniandtheweb commented 3 months ago

It already supports embeddings. You can call them with:

./sd ....  --embd-dir embedding_dir -p "prompt, embedding:embeddingname"
zhaojinzhou commented 3 months ago

It already supports embeddings. You can call them with:

./sd ....  --embd-dir embedding_dir -p "prompt, embedding:embeddingname"

but when I use badhand_v4.pt, It throws exception, and log error info : using checkpoint format It dost not work.

zhaojinzhou commented 3 months ago

It already supports embeddings. You can call them with:

./sd ....  --embd-dir embedding_dir -p "prompt, embedding:embeddingname"

5b98e7a1d21629738f537746530fc090672502e7_knock_capture_image

daniandtheweb commented 3 months ago

Can you send your full launch code and / or log?

zhaojinzhou commented 2 months ago

Can you send your full launch code and / or log?

[INFO ] stable-diffusion.cpp:501 - loading model from 'bWFqaWN2Ni5zYWZldGVuc29ycwo=' completed, taking 0.42s [INFO ] stable-diffusion.cpp:528 - running in eps-prediction mode [INFO ] stable-diffusion.cpp:655 - Attempting to apply 0 LoRAs [INFO ] stable-diffusion.cpp:1132 - apply_loras completed, taking 0.00s [INFO ] model.cpp:796 - load embeddings/FastNegativeV2.pt using checkpoint format /root/stable-diffusion.cpp/ggml/src/ggml-cuda/concat.cu:157: GGML_ASSERT(src0->type == GGML_TYPE_F32) failed Aborted (core dumped) root@bc413b608e61:~# ls embeddings/ FastNegativeV2.pt bad-picture-chill-75v.pt badhandv4.pt ng_deepnegative_v1_75t.pt root@bc413b608e61:~# ./trainer_cpp -m bWFqaWN2Ni5zYWZldGVuc29ycwo\= -p 'sexy girl 20_years_old' -n 'bad hands, FastNegativeV2' --steps 20 --embd-dir embeddings

aa5c501e1c909ad6e73ccac032c25e3584b4a7d7_knock_capture_image

zhaojinzhou commented 2 months ago

Can you send your full launch code and / or log?

[INFO ] stable-diffusion.cpp:501 - loading model from 'bWFqaWN2Ni5zYWZldGVuc29ycwo=' completed, taking 0.42s [INFO ] stable-diffusion.cpp:528 - running in eps-prediction mode [INFO ] stable-diffusion.cpp:655 - Attempting to apply 0 LoRAs [INFO ] stable-diffusion.cpp:1132 - apply_loras completed, taking 0.00s [INFO ] model.cpp:796 - load embeddings/FastNegativeV2.pt using checkpoint format /root/stable-diffusion.cpp/ggml/src/ggml-cuda/concat.cu:157: GGML_ASSERT(src0->type == GGML_TYPE_F32) failed Aborted (core dumped) root@bc413b608e61:~# ls embeddings/ FastNegativeV2.pt bad-picture-chill-75v.pt badhandv4.pt ng_deepnegative_v1_75t.pt root@bc413b608e61:~# ./trainer_cpp -m bWFqaWN2Ni5zYWZldGVuc29ycwo= -p 'sexy girl 20_years_old' -n 'bad hands, FastNegativeV2' --steps 20 --embd-dir embeddings

aa5c501e1c909ad6e73ccac032c25e3584b4a7d7_knock_capture_image

cublas version

daniandtheweb commented 1 month ago

I've been using the vulkan version before and there it worked but I'm now on the cublas version and you're right, there this fails. Apparently the needed operations to handle the embeddings still haven't been added.

However the issue isn't about the .pt format, I think it's about embeddings in general as also easynegative.safetensors doesn't work. I haven't been able to use any embedding for now on cublas.