search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
135.28k stars 25.83k forks source link

EMBEDED TRAINING FINISHED 0 steps. #7381

Open DaddyMow opened 1 year ago

DaddyMow commented 1 year ago

Is there an existing issue for this?

What happened?

when i click train embeddings for first few second it say's Preparing dataset from /content/stable-diffusion-webui/zpsz.. Screenshot 2023-01-30 030252

this is my dataset directory

dataset

Then after a few second's it stop's automatically Training] finished at 0 steps. Embedding saved to /content/stable-diffusion-webui/embeddings/grv.pt Screenshot 2023-01-30 030205

( i also restarted colab notebook many time but nothing happens looked up at similar issues like this but didn't work

Steps to reproduce the problem

  1. Go to .... Create embedding
  2. Preprocess images..
  3. ... Train embedding

What should have happened?

it should have trained dataset

Commit where the problem happens

Latest version/repo/commit

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Brave

Command Line Arguments

i am using protogen_v2_2_webui_colab.ipynb by camenduru

List of extensions

just normAL STUFF

Console logs

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.0/134.0 MB 8.7 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.7/18.7 MB 21.4 MB/s eta 0:00:00
Cloning into 'stable-diffusion-webui'...
remote: Enumerating objects: 15706, done.
remote: Counting objects: 100% (267/267), done.
remote: Compressing objects: 100% (149/149), done.
remote: Total 15706 (delta 157), reused 183 (delta 117), pack-reused 15439
Receiving objects: 100% (15706/15706), 27.11 MiB | 12.08 MiB/s, done.
Resolving deltas: 100% (10986/10986), done.
--2023-01-29 20:12:58--  https://raw.githubusercontent.com/camenduru/stable-diffusion-webui-scripts/main/run_n_times.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 533 [text/plain]
Saving to: ‘/content/stable-diffusion-webui/scripts/run_n_times.py’

/content/stable-dif 100%[===================>]     533  --.-KB/s    in 0s      

2023-01-29 20:12:58 (34.3 MB/s) - ‘/content/stable-diffusion-webui/scripts/run_n_times.py’ saved [533/533]

Cloning into '/content/stable-diffusion-webui/extensions/deforum-for-automatic1111-webui'...
remote: Enumerating objects: 53, done.
remote: Counting objects: 100% (53/53), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 53 (delta 2), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (53/53), 127.92 KiB | 1.58 MiB/s, done.
Cloning into '/content/stable-diffusion-webui/extensions/stable-diffusion-webui-images-browser'...
remote: Enumerating objects: 143, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 143 (delta 8), reused 6 (delta 6), pack-reused 129
Receiving objects: 100% (143/143), 39.48 KiB | 13.16 MiB/s, done.
Resolving deltas: 100% (50/50), done.
Cloning into '/content/stable-diffusion-webui/extensions/stable-diffusion-webui-huggingface'...
remote: Enumerating objects: 50, done.
remote: Counting objects: 100% (50/50), done.
remote: Compressing objects: 100% (42/42), done.
remote: Total 50 (delta 12), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (50/50), 11.69 KiB | 1.46 MiB/s, done.
Cloning into '/content/stable-diffusion-webui/extensions/sd-civitai-browser'...
remote: Enumerating objects: 47, done.
remote: Counting objects: 100% (47/47), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 47 (delta 10), reused 34 (delta 7), pack-reused 0
Unpacking objects: 100% (47/47), 12.72 KiB | 1.41 MiB/s, done.
Cloning into '/content/stable-diffusion-webui/extensions/sd-webui-additional-networks'...
remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 20 (delta 4), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (20/20), 15.19 KiB | 3.80 MiB/s, done.
/content/stable-diffusion-webui
--2023-01-29 20:13:00--  https://huggingface.co/ckpt/Protogen_V2.2/resolve/main/Protogen_V2.2.ckpt
Resolving huggingface.co (huggingface.co)... 54.235.118.239, 3.231.67.228, 2600:1f18:147f:e850:e203:c458:10cd:fc3c, ...
Connecting to huggingface.co (huggingface.co)|54.235.118.239|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn-lfs.huggingface.co/repos/2f/ea/2feacd6fed3958fdee1a8537fdd431cb1d4f293425f03ed60118a6e8d717139a/9a428ee39f591a34006b1661fab76942e293666fbac0a109aef97bec9d542cf1?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27Protogen_V2.2.ckpt%3B+filename%3D%22Protogen_V2.2.ckpt%22%3B&Expires=1675282381&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zLzJmL2VhLzJmZWFjZDZmZWQzOTU4ZmRlZTFhODUzN2ZkZDQzMWNiMWQ0ZjI5MzQyNWYwM2VkNjAxMThhNmU4ZDcxNzEzOWEvOWE0MjhlZTM5ZjU5MWEzNDAwNmIxNjYxZmFiNzY5NDJlMjkzNjY2ZmJhYzBhMTA5YWVmOTdiZWM5ZDU0MmNmMT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2NzUyODIzODF9fX1dfQ__&Signature=FIotYe4E41MIkLL97jkCUTLXI9tbUmhgQTj522iW7YnR-wt3nI3YaEyiH2YTIY7covR5N7TVitacEr7aRHjYd-DUacT6KiBOygPsTEVcl6KCLBfmYXAOGFw4jaMcPkuKsYpEsLTq73iT1vSA%7EnedSkWecoKEiDQqrIlCFZDERBrQQfwxKM1Ls24nB4Ns1jwCzdJFB-b6fkXymmQ9hscmlotYAiabaVv4oOOeWQEgw2Tl3pa-OC6%7EmrkOa5sQDOqzTYx4ibZ1dV4h2T16TlJz38BWfCq3%7EqTATr3bKkSbuDGW00JXmz2XOSfTzs7H5R%7EEWSSb1x8yK9-RunGivqOuNA__&Key-Pair-Id=KVTP0A1DKRTAX [following]
--2023-01-29 20:13:00--  https://cdn-lfs.huggingface.co/repos/2f/ea/2feacd6fed3958fdee1a8537fdd431cb1d4f293425f03ed60118a6e8d717139a/9a428ee39f591a34006b1661fab76942e293666fbac0a109aef97bec9d542cf1?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27Protogen_V2.2.ckpt%3B+filename%3D%22Protogen_V2.2.ckpt%22%3B&Expires=1675282381&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zLzJmL2VhLzJmZWFjZDZmZWQzOTU4ZmRlZTFhODUzN2ZkZDQzMWNiMWQ0ZjI5MzQyNWYwM2VkNjAxMThhNmU4ZDcxNzEzOWEvOWE0MjhlZTM5ZjU5MWEzNDAwNmIxNjYxZmFiNzY5NDJlMjkzNjY2ZmJhYzBhMTA5YWVmOTdiZWM5ZDU0MmNmMT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2NzUyODIzODF9fX1dfQ__&Signature=FIotYe4E41MIkLL97jkCUTLXI9tbUmhgQTj522iW7YnR-wt3nI3YaEyiH2YTIY7covR5N7TVitacEr7aRHjYd-DUacT6KiBOygPsTEVcl6KCLBfmYXAOGFw4jaMcPkuKsYpEsLTq73iT1vSA%7EnedSkWecoKEiDQqrIlCFZDERBrQQfwxKM1Ls24nB4Ns1jwCzdJFB-b6fkXymmQ9hscmlotYAiabaVv4oOOeWQEgw2Tl3pa-OC6%7EmrkOa5sQDOqzTYx4ibZ1dV4h2T16TlJz38BWfCq3%7EqTATr3bKkSbuDGW00JXmz2XOSfTzs7H5R%7EEWSSb1x8yK9-RunGivqOuNA__&Key-Pair-Id=KVTP0A1DKRTAX
Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 52.222.174.30, 52.222.174.3, 52.222.174.26, ...
Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|52.222.174.30|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4265335100 (4.0G) [binary/octet-stream]
Saving to: ‘/content/stable-diffusion-webui/models/Stable-diffusion/Protogen_V2.2.ckpt’

/content/stable-dif 100%[===================>]   3.97G   109MB/s    in 36s     

2023-01-29 20:13:36 (113 MB/s) - ‘/content/stable-diffusion-webui/models/Stable-diffusion/Protogen_V2.2.ckpt’ saved [4265335100/4265335100]

sed: can't read /content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/util.py: No such file or directory
Python 3.8.10 (default, Nov 14 2022, 12:59:47) 
[GCC 9.4.0]
Commit hash: 4af3ca5393151d61363c30eef4965e694eeac15e
Installing gfpgan
Installing clip
Installing open_clip
Cloning Stable Diffusion into repositories/stable-diffusion-stability-ai...
Cloning Taming Transformers into repositories/taming-transformers...
Cloning K-diffusion into repositories/k-diffusion...
Cloning CodeFormer into repositories/CodeFormer...
Cloning BLIP into repositories/BLIP...
Installing requirements for CodeFormer
Installing requirements for Web UI

Launching Web UI with arguments: --share --xformers --enable-insecure-extension-access
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Downloading: 100% 939k/939k [00:00<00:00, 1.88MB/s]
Downloading: 100% 512k/512k [00:00<00:00, 1.52MB/s]
Downloading: 100% 389/389 [00:00<00:00, 406kB/s]
Downloading: 100% 905/905 [00:00<00:00, 1.01MB/s]
Downloading: 100% 4.41k/4.41k [00:00<00:00, 4.66MB/s]
Downloading: 100% 1.59G/1.59G [00:21<00:00, 81.4MB/s]
Loading weights [54d316c4] from /content/stable-diffusion-webui/models/Stable-diffusion/Protogen_V2.2.ckpt
Applying xformers cross attention optimization.
Model loaded.
Loaded a total of 0 textual inversion embeddings.
Embeddings: 
Running on local URL:  http://127.0.0.1:7860
Running on public URL: https://c87c9580dabad9ad.gradio.app

This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces
Loaded a total of 1 textual inversion embeddings.
Embeddings: grv
Applying xformers cross attention optimization.
Error completing request
Arguments: ('grv', '0.005', 1, 1, '/content/MyDrive/grv/test', 'textual_inversion', 512, 512, 100000, True, 0, 'random', 500, 500, '/content/stable-diffusion-webui/textual_inversion_templates/style_filewords.txt', True, True, '', '', 20, 0, 7, -1.0, 512, 512) {}
Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/call_queue.py", line 45, in f
    res = list(func(*args, **kwargs))
  File "/content/stable-diffusion-webui/modules/call_queue.py", line 28, in f
    res = func(*args, **kwargs)
  File "/content/stable-diffusion-webui/modules/textual_inversion/ui.py", line 33, in train_embedding
    embedding, filename = modules.textual_inversion.textual_inversion.train_embedding(*args)
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 231, in train_embedding
    validate_train_inputs(embedding_name, learn_rate, batch_size, gradient_step, data_root, template_file, steps, save_embedding_every, create_image_every, log_directory, name="embedding")
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 214, in validate_train_inputs
    assert os.path.isdir(data_root), "Dataset directory doesn't exist"
AssertionError: Dataset directory doesn't exist

Applying xformers cross attention optimization.
Error completing request
Arguments: ('grv', '0.005', 1, 1, '/content/MyDrive/grv/test', 'textual_inversion', 512, 512, 100000, True, 0, 'random', 500, 500, '/content/stable-diffusion-webui/textual_inversion_templates/style_filewords.txt', True, True, '', '', 20, 0, 7, -1.0, 512, 512) {}
Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/call_queue.py", line 45, in f
    res = list(func(*args, **kwargs))
  File "/content/stable-diffusion-webui/modules/call_queue.py", line 28, in f
    res = func(*args, **kwargs)
  File "/content/stable-diffusion-webui/modules/textual_inversion/ui.py", line 33, in train_embedding
    embedding, filename = modules.textual_inversion.textual_inversion.train_embedding(*args)
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 231, in train_embedding
    validate_train_inputs(embedding_name, learn_rate, batch_size, gradient_step, data_root, template_file, steps, save_embedding_every, create_image_every, log_directory, name="embedding")
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 214, in validate_train_inputs
    assert os.path.isdir(data_root), "Dataset directory doesn't exist"
AssertionError: Dataset directory doesn't exist

Downloading: "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_caption_capfilt_large.pth" to /content/stable-diffusion-webui/models/BLIP/model_base_caption_capfilt_large.pth

100% 855M/855M [00:21<00:00, 41.3MB/s]
Downloading: 100% 226k/226k [00:00<00:00, 679kB/s]
Downloading: 100% 28.0/28.0 [00:00<00:00, 28.2kB/s]
Downloading: 100% 570/570 [00:00<00:00, 693kB/s]
load checkpoint from /content/stable-diffusion-webui/models/BLIP/model_base_caption_capfilt_large.pth
100%|███████████████████████████████████████| 890M/890M [00:18<00:00, 51.5MiB/s]
Downloading: "https://github.com/AUTOMATIC1111/TorchDeepDanbooru/releases/download/v1/model-resnet_custom_v3.pt" to /content/stable-diffusion-webui/models/torch_deepdanbooru/model-resnet_custom_v3.pt

100% 614M/614M [00:33<00:00, 19.5MB/s]
  0% 0/13 [00:00<?, ?it/s]downloading face detection model from 'https://github.com/opencv/opencv_zoo/blob/91fb0290f50896f38a0ab1e558b74b16bc009428/models/face_detection_yunet/face_detection_yunet_2022mar.onnx?raw=true' to '/content/stable-diffusion-webui/models/opencv/face_detection_yunet.onnx'
  8% 1/13 [00:12<02:26, 12.22s/it]
Training at rate of 0.005 until step 100000
Preparing dataset...
100% 104/104 [00:08<00:00, 12.77it/s]
  0% 0/100000 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
[1.0, 2.0, 1.0]
Activation function is linear
Weight initialization is Normal
Layer norm is set to False
Dropout usage is set to True
Activate last layer is set to False
Optimizer name is AdamW
No saved optimizer exists in checkpoint
Training at rate of 1e-05 until step 100000
Preparing dataset...
100% 104/104 [00:06<00:00, 16.96it/s]
  0% 0/100000 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/hypernetworks/hypernetwork.py", line 535, in train_hypernetwork
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 100000
Preparing dataset...
100% 104/104 [00:07<00:00, 14.73it/s]
  0% 0/100000 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 99999
Preparing dataset...
100% 104/104 [00:07<00:00, 14.76it/s]
  0% 0/99999 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 99999
Preparing dataset...
100% 104/104 [00:07<00:00, 14.67it/s]
  0% 0/99999 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 99999
Preparing dataset...
100% 104/104 [00:06<00:00, 17.02it/s]
  0% 0/99999 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 99999
Preparing dataset...
100% 104/104 [00:06<00:00, 17.01it/s]
  0% 0/99999 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 99999
Preparing dataset...
100% 104/104 [00:06<00:00, 17.10it/s]
  0% 0/99999 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 99999
Preparing dataset...
100% 104/104 [00:06<00:00, 16.99it/s]
  0% 0/99999 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.
Training at rate of 0.005 until step 99999
Preparing dataset...
100% 104/104 [00:06<00:00, 16.93it/s]
  0% 0/99999 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding
    loss = shared.sd_model(x, c)[0] / gradient_step
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 846, in forward
    return self.p_losses(x, c, t, *args, **kwargs)
  File "/content/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 903, in p_losses
    logvar_t = self.logvar[t].to(self.device)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Applying xformers cross attention optimization.

Additional information

*I also tried on diffrent model sd1.5 , 2.1 and bunch of other in diff -diff colab notebook even in my personal one but it didn't work for me

magaspah commented 1 year ago

.

Mousewrites commented 1 year ago

There's a bug right now that if you have xformers turned on, and try to train a 1.5 embed with xformers, it 'trains' but it doesn't actually train anything. The training functions, looks at your prompts, but doesn't seem to actually reference your IMAGES at all. Here's more people talking about it: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/7264#issue-1559093472

Happens on my 3060, not just the card listed in the title of that bug.

Hopefully it is fixed soon.

Supposedly, 2.1 shouldn't be affected, but... I'm not sure it's working correctly either. Turning xformers off is apparently the 'workaround' but everything DRAGS when I do that.

EDIT: whoops, wrong window, was replying to a different bug. :)

gregtorn commented 1 year ago

is there any solution? tried to train the embending but get Training finished at 0 steps.

rayzheng1980 commented 1 year ago

same issue, waiting for solution.

Nik00888 commented 6 months ago

If this happens, I just run txt2img once (with some very simple prompt) on the vanilla model, and then it will work.