search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
140.58k stars 26.61k forks source link

[Bug]: Textual Inversion not being loaded #13830

Open JoaoPedroAbubakir opened 11 months ago

JoaoPedroAbubakir commented 11 months ago

Is there an existing issue for this?

What happened?

Textual inversions not loading properly. They show up in the first run, then do not for any subsequent run. However, from image result they are not used in either.

Steps to reproduce the problem

  1. Go to attempt to generate an image using the following Textual Inversion Embeddings: EasyNegative, negative_hand
  2. Generate First Image, TI hashes shows up in baked metadata
  3. Generate Second Image, it is the same as the first and no TI hashes baked in metadata

What should have happened?

Textual Inversions should have been used to generate the image consistently.

Sysinfo

sysinfo-2023-11-03-07-01.txt

What browsers do you use to access the UI ?

Other: Opera GX

Console logs

venv "D:\Stable Diffusion\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
2023-11-03 04:03:22 DEBUG [root] Installing put extensions here.txt
Launching Web UI with arguments:
2023-11-03 04:03:24 DEBUG [matplotlib] matplotlib data path: D:\Stable Diffusion\stable-diffusion-webui\venv\lib\site-packages\matplotlib\mpl-data
2023-11-03 04:03:24 DEBUG [matplotlib] CONFIGDIR=C:\Users\Argo\.matplotlib
2023-11-03 04:03:24 DEBUG [matplotlib] interactive is False
2023-11-03 04:03:24 DEBUG [matplotlib] platform is win32
2023-11-03 04:03:24 DEBUG [matplotlib] CACHEDIR=C:\Users\Argo\.matplotlib
2023-11-03 04:03:24 DEBUG [matplotlib.font_manager] Using fontManager instance from C:\Users\Argo\.matplotlib\fontlist-v330.json
2023-11-03 04:03:24 DEBUG [httpx] load_ssl_context verify=True cert=None trust_env=True http2=False
2023-11-03 04:03:24 DEBUG [httpx] load_verify_locations cafile='D:\\Stable Diffusion\\stable-diffusion-webui\\venv\\lib\\site-packages\\certifi\\cacert.pem'
2023-11-03 04:03:25 DEBUG [httpx] load_ssl_context verify=True cert=None trust_env=True http2=False
2023-11-03 04:03:25 DEBUG [httpx] load_verify_locations cafile='D:\\Stable Diffusion\\stable-diffusion-webui\\venv\\lib\\site-packages\\certifi\\cacert.pem'
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing BlpImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing BmpImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing BufrStubImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing CurImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing DcxImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing DdsImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing EpsImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing FitsImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing FitsStubImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing FliImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing FpxImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Image: failed to import FpxImagePlugin: No module named 'olefile'
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing FtexImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing GbrImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing GifImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing GribStubImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing Hdf5StubImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing IcnsImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing IcoImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing ImImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing ImtImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing IptcImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing JpegImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing Jpeg2KImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing McIdasImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing MicImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Image: failed to import MicImagePlugin: No module named 'olefile'
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing MpegImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing MpoImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing MspImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PalmImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PcdImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PcxImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PdfImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PixarImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PngImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PpmImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing PsdImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing QoiImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing SgiImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing SpiderImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing SunImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing TgaImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing TiffImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing WebPImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing WmfImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing XbmImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing XpmImagePlugin
2023-11-03 04:03:25 DEBUG [PIL.Image] Importing XVThumbImagePlugin
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
2023-11-03 04:03:26 DEBUG [git.cmd] Popen(['git', 'version'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=None)
2023-11-03 04:03:26 DEBUG [git.cmd] Popen(['git', 'version'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=None)
Loading weights [6292dd40d6] from D:\Stable Diffusion\stable-diffusion-webui\models\Stable-diffusion\meinapastel_v6Pastel.safetensors
2023-11-03 04:03:27 DEBUG [asyncio] Using selector: SelectSelector
2023-11-03 04:03:28 DEBUG [httpx] load_ssl_context verify=None cert=None trust_env=True http2=False
Running on local URL:  http://127.0.0.1:7860
2023-11-03 04:03:28 DEBUG [urllib3.connectionpool] Starting new HTTP connection (1): 127.0.0.1:7860
2023-11-03 04:03:28 DEBUG [urllib3.connectionpool] http://127.0.0.1:7860 "GET /startup-events HTTP/1.1" 200 5
2023-11-03 04:03:28 DEBUG [urllib3.connectionpool] Starting new HTTP connection (1): 127.0.0.1:7860
2023-11-03 04:03:28 DEBUG [urllib3.connectionpool] http://127.0.0.1:7860 "HEAD / HTTP/1.1" 200 0

To create a public link, set `share=True` in `launch()`.
Creating model from config: D:\Stable Diffusion\stable-diffusion-webui\configs\v1-inference.yaml
Startup time: 7.3s (prepare environment: 1.6s, import torch: 2.4s, import gradio: 0.7s, setup paths: 0.6s, initialize shared: 0.2s, other imports: 0.4s, load scripts: 0.7s, create ui: 0.4s, gradio launch: 0.3s).
2023-11-03 04:03:29 DEBUG [matplotlib.pyplot] Loaded backend tkagg version 8.6.
2023-11-03 04:03:29 DEBUG [matplotlib.pyplot] Loaded backend agg version v2.2.
2023-11-03 04:03:29 DEBUG [matplotlib.pyplot] Loaded backend TkAgg version 8.6.
2023-11-03 04:03:29 DEBUG [matplotlib.pyplot] Loaded backend agg version v2.2.
2023-11-03 04:03:29 DEBUG [matplotlib.pyplot] Loaded backend TkAgg version 8.6.
2023-11-03 04:03:29 DEBUG [matplotlib.pyplot] Loaded backend agg version v2.2.
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 DEBUG [git.cmd] Popen(['git', 'remote', 'get-url', '--all', 'origin'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=None)
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 DEBUG [git.cmd] Popen(['git', 'cat-file', '--batch-check'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=<valid stream>)
2023-11-03 04:03:30 DEBUG [git.cmd] Popen(['git', 'cat-file', '--batch'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=<valid stream>)
Applying attention optimization: Doggettx... done.
2023-11-03 04:03:30 DEBUG [git.cmd] Popen(['git', 'remote', 'get-url', '--all', 'origin'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=None)
2023-11-03 04:03:30 DEBUG [git.cmd] Popen(['git', 'cat-file', '--batch-check'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=<valid stream>)
2023-11-03 04:03:30 DEBUG [git.cmd] Popen(['git', 'cat-file', '--batch'], cwd=D:\Stable Diffusion\stable-diffusion-webui, universal_newlines=False, shell=None, istream=<valid stream>)
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
Model loaded in 2.8s (load weights from disk: 0.6s, create model: 0.3s, apply weights to model: 1.6s, calculate empty prompt: 0.1s).
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:30 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:37 DEBUG [PIL.PngImagePlugin] STREAM b'IHDR' 16 13
2023-11-03 04:03:37 DEBUG [PIL.PngImagePlugin] STREAM b'tEXt' 41 647
2023-11-03 04:03:37 DEBUG [PIL.PngImagePlugin] STREAM b'IDAT' 700 65536
2023-11-03 04:03:37 DEBUG [PIL.PngImagePlugin] STREAM b'eXIf' 1008421 14
2023-11-03 04:03:37 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:37 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:37 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:37 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:38 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:38 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:38 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:38 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:38 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:38 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:39 INFO [modules.shared_state] Starting job task(1ihzxqobwt3ma96)
2023-11-03 04:03:39 DEBUG [modules.sd_hijack_clip] easynegative
2023-11-03 04:03:39 DEBUG [modules.sd_hijack_clip] negative_hand-neg
100%|██████████████████████████████████████████████████████████████████████████████████| 45/45 [00:16<00:00,  2.69it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 45/45 [00:15<00:00,  2.87it/s]
2023-11-03 04:03:57 INFO [modules.shared_state] Ending job task(1ihzxqobwt3ma96) (18.21 seconds)00:15<00:00,  3.45it/s]
2023-11-03 04:03:57 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:03:57 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
2023-11-03 04:03:59 INFO [modules.shared_state] Starting job task(16ppy8q6e2rzfq6)
100%|██████████████████████████████████████████████████████████████████████████████████| 45/45 [00:15<00:00,  2.83it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 45/45 [00:15<00:00,  2.85it/s]
2023-11-03 04:04:16 INFO [modules.shared_state] Ending job task(16ppy8q6e2rzfq6) (16.25 seconds)00:15<00:00,  3.50it/s]
2023-11-03 04:04:16 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 200 OK"
2023-11-03 04:04:16 INFO [httpx] HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"

Additional information

I have no idea where to log when Embeddings are loaded so I added them to modules.sd_hijack_clip:234, when metadata is generated. You can see from the log that on the first pass it at least showed up, but on the second it dit not.

JoaoPedroAbubakir commented 11 months ago

What I have tried:

  1. Fresh Autmatic1111 install
  2. Install in another Hard Drive as suggested in another issue
  3. Fresh windows install on formatted Hard Drive,
  4. Standalone Automatic1111 from release page with embedded Python and Git
  5. Automatic1111 cloned from github and run just webui-user.bat with manually installed Python
  6. Running prompt with curl/postman api call
  7. using (embedding:name:1.1) in prompt as suggested in a reddit post

Some other additional info: Nvidia driver version - 546.01

JoaoPedroAbubakir commented 11 months ago

After many hours of discussion with CivitAI discord, we have found a workaround. Using aDetail helps with keeping TI info in metadata and seems to force the loading.

eterniti commented 10 months ago

I have this bug aswell. Happens to other embeddings too.

NoYzE1 commented 9 months ago

This is most likely a cache problem. If you change something in the promt, the embedding hash will show again, but every consecutive run it will not.

I fixed this locally by disabling the caching in processing.py inside the function get_conds_with_caching(...) by commenting out the lines

for cache in caches:
    if cache[0] is not None and cached_params == cache[0]:
        return cache[1]

which is probably not optimal, but this also fixes embedding training preview when using text2img promt for me, so it's better than to have this cache since i don't see any negative performance impact or problems with it being disabled.