Create Embedding is Non-functional

OttCS commented 1 month ago

Checklist

[X] The issue exists after disabling all extensions
[X] The issue exists on a clean installation of webui
[ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[X] The issue exists in the current version of the webui
[X] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

Creating an Embedding results in an error and a broken file that webui doesn't recognize in the training tab.

Steps to reproduce the problem

Fresh install of webui
Go to Train -> Create Embedding
Enter any name and click create
Error appears in log, despite creating the file in the embeddings folder

What should have happened?

Create an empty embedding file without error, and be able recognize and train it in the Train tab.

What browsers do you use to access the UI ?

Google Chrome

Sysinfo

sysinfo.txt

Console logs

################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye), Fedora 34+ and openSUSE Leap 15.4 or newer.
################################################################

################################################################
Running on user user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Experimental support for Renoir: make sure to have at least 4GB of VRAM and 10GB of RAM or enable cpu mode: --use-cpu all --no-half
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Launching launch.py...
################################################################
glibc version is 2.39
Check TCMalloc: libtcmalloc_minimal.so.4
libtcmalloc_minimal.so.4 is linked with libc.so,execute LD_PRELOAD=/lib64/libtcmalloc_minimal.so.4
Python 3.10.14 (main, Apr 17 2024, 00:00:00) [GCC 14.0.1 20240411 (Red Hat 14.0.1-0)]
Version: v1.9.4
Commit hash: feee37d75f1b168768014e4634dcb156ee649c05
Launching Web UI with arguments: --xformers
Loading weights [6ce0161689] from /mnt/Games/trainer-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
Creating model from config: /mnt/Games/trainer-webui/configs/v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
/mnt/Games/trainer-webui/venv/lib64/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Startup time: 8.4s (prepare environment: 1.6s, import torch: 2.7s, import gradio: 0.6s, setup paths: 1.3s, initialize shared: 0.2s, other imports: 0.5s, load scripts: 0.6s, create ui: 0.3s, gradio launch: 0.6s).
Opening in existing browser session.
Applying attention optimization: xformers... done.
Model loaded in 3.3s (load weights from disk: 0.5s, create model: 0.6s, apply weights to model: 1.9s).
*** Error verifying pickled file from /mnt/Games/trainer-webui/embeddings/bugged.pt
*** The file may be malicious, so the program is not going to read it.
*** You can skip this check with --disable-safe-unpickle commandline argument.
*** 
    Traceback (most recent call last):
      File "/mnt/Games/trainer-webui/modules/safe.py", line 137, in load_with_extra
        check_pt(filename, extra_handler)
      File "/mnt/Games/trainer-webui/modules/safe.py", line 84, in check_pt
        check_zip_filenames(filename, z.namelist())
      File "/mnt/Games/trainer-webui/modules/safe.py", line 76, in check_zip_filenames
        raise Exception(f"bad file inside {filename}: {name}")
    Exception: bad file inside /mnt/Games/trainer-webui/embeddings/bugged.pt: bugged/byteorder

---
*** Error loading embedding bugged.pt
    Traceback (most recent call last):
      File "/mnt/Games/trainer-webui/modules/textual_inversion/textual_inversion.py", line 203, in load_from_dir
        self.load_from_file(fullfn, fn)
      File "/mnt/Games/trainer-webui/modules/textual_inversion/textual_inversion.py", line 184, in load_from_file
        embedding = create_embedding_from_data(data, name, filename=filename, filepath=path)
      File "/mnt/Games/trainer-webui/modules/textual_inversion/textual_inversion.py", line 284, in create_embedding_from_data
        if 'string_to_param' in data:  # textual inversion embeddings
    TypeError: argument of type 'NoneType' is not iterable

---

Additional information

Screenshot from 2024-06-07 01-05-58 Create Embedding is Clicked

Screenshot from 2024-06-07 01-06-24 Error appears in console

Screenshot from 2024-06-07 01-06-40 Despite the error, a file is created

Screenshot from 2024-06-07 01-07-26 The file is not recognized within the Train tab, though other downloaded embeddings can appear

missionfloyd commented 1 month ago

The train tab is outdated. Use this instead. https://github.com/bmaltais/kohya_ss

OttCS commented 1 month ago

If the train tab is outdated/doesn't work it's still an existing issue with Automatic1111's webui. Also, kohya_ss has further issues on linux so I'd prefer to help get it working in this project.

MIKETUFFIN66 commented 1 month ago

i am having the same problem, please it would be kind if it gets updated and functionally working

MIKETUFFIN66 commented 1 month ago

I JUST FOUND A SOLUTION THAT WILL READ ALL THE EMBEDDING CREATED

"set COMMANDLINE_ARGS=--disable-safe-unpickle"

Here: ..\stable-diffusion-webui\webui-user.bat

DO THIS AND IT WILL WORK thks

OttCS commented 1 month ago

Awesome! I'm going to keep this issue open, because by default I believe Stable Diffusion shouldn't be creating files that get flagged as "unsafe" and the error printed to the log still needs to be addressed.

MIKETUFFIN66 commented 1 month ago

Awesome! I'm going to keep this issue open, because by default I believe Stable Diffusion shouldn't be creating files that get flagged as "unsafe" and the error printed to the log still needs to be addressed.

now there is another problem , when you "set COMMANDLINE_ARGS=--disable-safe-unpickle" , it allows you to create the embedding, then you put it in the SD to try .. but only this time i deleted --disable-safe-unpickle command from the argument.. the embedding did not work !!! meaning even if you create an embedding and train it then try it with out the "set COMMANDLINE_ARGS=--disable-safe-unpickle" ( to be safe) .. its not GONNA WORK !!

this issue has to be solved yes you are right.

AUTOMATIC1111 / stable-diffusion-webui