search

d8ahazard / sd_dreambooth_extension

Other
1.86k stars 281 forks source link

UnboundLocalError: local variable 'g_cuda' referenced before assignment #479

Closed i0ntempest closed 1 year ago

i0ntempest commented 1 year ago

Please find the following lines in the console and paste them below. If you do not provide this information, your issue will be automatically closed.

Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Commit hash: 59c2dfe1e6a29dcf231fa9d894ac238887e40c39
Installing requirements for Web UI

#######################################################################################################
Initializing Dreambooth
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Dreambooth revision: 82f7c9e3d85be3ea8c32334dd7efbba740279f51
SD-WebUI revision: 59c2dfe1e6a29dcf231fa9d894ac238887e40c39

Checking Dreambooth requirements...
[+] bitsandbytes version 0.35.0 installed.
[+] diffusers version 0.10.2 installed.
[+] transformers version 4.25.1 installed.
[+] xformers version 0.0.14.dev0 installed.
[+] torch version 1.12.1+cu116 installed.
[+] torchvision version 0.13.1+cu116 installed.
#######################################################################################################

Have you read the Readme? Yes Have you completely restarted the stable-diffusion-webUI, not just reloaded the UI? Yes Have you updated Dreambooth to the latest revision? Yes Have you updated the Stable-Diffusion-WebUI to the latest version? Yes No, really. Please save us both some trouble and update the SD-WebUI and Extension and restart before posting this. Reply 'OK' Below to acknowledge that you did this. OK Describe the bug During the cleanup after saving weights (line 942), g_cuda is referenced before assignment in the if statement, causing training to be interrupted. I wrapped it with a try and it looks like it does not affect the actual training.

Provide logs This log contains an unrelated CUDA OOM error before the actual error I want to report.

Dreambooth API layer loaded
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading weights [e6e8e1fc] from G:\Program Files\stable-diffusion-webui\models\Stable-diffusion\animefull-latest.ckpt
Applying cross attention optimization (Doggettx).
Model loaded.
Loaded a total of 61 textual inversion embeddings.
Embeddings: [masked]
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Concept 0 class dir is G:\Program Files\stable-diffusion-webui\models\dreambooth\kobaryo_mdkb\classifiers_0
Starting Dreambooth training...
 Allocated 0.0/2.0GB
 Reserved: 0.0/2.0GB

Initializing dreambooth training...
Patching transformers to fix kwargs errors.
G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\transformers\generation_utils.py:24: FutureWarning: Importing `GenerationMixin` from `src/transformers/generation_utils.py` is deprecated and will be removed in Transformers v5. Import as `from transformers import GenerationMixin` instead.
  warnings.warn(
Replace CrossAttention.forward to use default
Checking concept: {'max_steps': -1, 'instance_data_dir': 'D:\\Downloads\\Kobaryo_mdkb DreamBooth\\Train', 'class_data_dir': '', 'instance_prompt': 'by mdkb', 'class_prompt': '', 'save_sample_prompt': '(((extremely detailed unity 8K CG))), ((masterpiece)), (best quality), (an extremely delicate and beautiful), (intricate detail), (super delicate), illustration, hires, absurdres, highly detailed, rich detailed, beautiful detailed eyes, ((1girl)), ', 'save_sample_template': '', 'instance_token': '', 'class_token': '', 'num_class_images': 0, 'class_negative_prompt': '', 'class_guidance_scale': 7.5, 'class_infer_steps': 40, 'save_sample_negative_prompt': 'polar lowres, lowres, ((bad anatomy)), ((bad hands)), (bad body), bad proportions, gross proportions, text, error, missing fingers, (missing arms), (missing legs), extra digit, (extra fingers), fewer digits,\xa0(Mutated hands and fingers:1.5), one hand with more than 5 fingers, one hand with less than 5 fingers,  (extra limbs), (extra arms), (extra legs), (missing limbs), (missing legs), malformed limbs, fused fingers, (too many fingers), long neck, cross-eyed, mutated hands, cropped, poorly drawn hands, poorly drawn face, mutation, deformed, deformed legs, more than two knee, (bad feet), malformed feet, extra feet, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, fat, ugly, morbid, poorly drawn asymmetric eyes,\xa0bad shoes, fused shoes, poorly drawn shoes, bad gloves, poorly drawn gloves, fused gloves, bad hairs, poorly drawn hairs, fused hairs,bad eyes, fused eyes, poorly drawn eyes, extra eyes, [blurry], disfigured, bad mouth, fused mouth, poorly drawn mouth, dirty face, dirty teeth, humpbacked,\xa0', 'n_save_sample': 1, 'sample_seed': -1, 'save_guidance_scale': 10, 'save_infer_steps': 35}
Concept requires 0 images.
 Loaded model.
 Allocated: 0.0GB
 Reserved: 0.0GB

Using magick windows DLL!
CUDA SETUP: Loading binary G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cudaall.dll...
 Scheduler, EMA Loaded.
 Allocated: 3.8GB
 Reserved: 3.9GB

***** Running training *****
  Num examples = 83
  Num batches each epoch = 83
  Num Epochs = 126
  Instantaneous batch size per device = 1
  Total train batch size (w. parallel, distributed & accumulation) = 1
  Gradient Accumulation steps = 1
  Total optimization steps = 10400
  Actual steps: 10400
   Training settings: CPU: False Adam: True, Prec: fp16, Grad: True, TextTr: True EM: False, LR: 2e-06 LORA:False
 Allocated: 3.8GB
 Reserved: 3.9GB

Steps:   0%|                                          | 0/10400 [00:09<?, ?it/s, loss=0.00601, lr=2e-6, vram=5.7/9.5GB] Step 0 completed.
 Allocated: 5.7GB
 Reserved: 9.5GB

Steps:   0%|                               | 2/10400 [00:12<16:28:55,  5.71s/it, loss=0.00601, lr=2e-6, vram=5.7/9.5GB] Exception while training: CUDA out of memory. Tried to allocate 512.00 MiB (GPU 0; 11.99 GiB total capacity; 10.22 GiB already allocated; 0 bytes free; 11.28 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
 Allocated: 9.5GB
 Reserved: 11.3GB

Traceback (most recent call last):
  File "G:\Program Files\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\train_dreambooth.py", line 1030, in main
    accelerator.backward(loss)
  File "G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\accelerate\accelerator.py", line 1188, in backward
    self.scaler.scale(loss).backward(**kwargs)
  File "G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\function.py", line 253, in apply
    return user_fn(self, *args)
  File "G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\utils\checkpoint.py", line 146, in backward
    torch.autograd.backward(outputs_with_grad, args_with_grad)
  File "G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: CUDA out of memory. Tried to allocate 512.00 MiB (GPU 0; 11.99 GiB total capacity; 10.22 GiB already allocated; 0 bytes free; 11.28 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
|===========================================================================|
|                  PyTorch CUDA memory summary, device ID 0                 |
|---------------------------------------------------------------------------|
|            CUDA OOMs: 1            |        cudaMalloc retries: 4         |
|===========================================================================|
|        Metric         | Cur Usage  | Peak Usage | Tot Alloc  | Tot Freed  |
|---------------------------------------------------------------------------|
| Allocated memory      |    9689 MB |   10973 MB |  213624 MB |  203934 MB |
|       from large pool |    9434 MB |   10721 MB |  210636 MB |  201202 MB |
|       from small pool |     255 MB |     307 MB |    2987 MB |    2732 MB |
|---------------------------------------------------------------------------|
| Active memory         |    9689 MB |   10973 MB |  213624 MB |  203934 MB |
|       from large pool |    9434 MB |   10721 MB |  210636 MB |  201202 MB |
|       from small pool |     255 MB |     307 MB |    2987 MB |    2732 MB |
|---------------------------------------------------------------------------|
| GPU reserved memory   |   11548 MB |   11564 MB |   27224 MB |   15676 MB |
|       from large pool |   11272 MB |   11272 MB |   26710 MB |   15438 MB |
|       from small pool |     276 MB |     318 MB |     514 MB |     238 MB |
|---------------------------------------------------------------------------|
| Non-releasable memory |    1858 MB |    4841 MB |  275128 MB |  273270 MB |
|       from large pool |    1837 MB |    4840 MB |  272016 MB |  270179 MB |
|       from small pool |      20 MB |      51 MB |    3112 MB |    3091 MB |
|---------------------------------------------------------------------------|
| Allocations           |    4418    |    4610    |   30650    |   26232    |
|       from large pool |     893    |     957    |   11682    |   10789    |
|       from small pool |    3525    |    3654    |   18968    |   15443    |
|---------------------------------------------------------------------------|
| Active allocs         |    4418    |    4610    |   30650    |   26232    |
|       from large pool |     893    |     957    |   11682    |   10789    |
|       from small pool |    3525    |    3654    |   18968    |   15443    |
|---------------------------------------------------------------------------|
| GPU reserved segments |     356    |     376    |     568    |     212    |
|       from large pool |     218    |     218    |     311    |      93    |
|       from small pool |     138    |     159    |     257    |     119    |
|---------------------------------------------------------------------------|
| Non-releasable allocs |     201    |     281    |   14533    |   14332    |
|       from large pool |      73    |     124    |    7008    |    6935    |
|       from small pool |     128    |     184    |    7525    |    7397    |
|---------------------------------------------------------------------------|
| Oversize allocations  |       0    |       0    |       0    |       0    |
|---------------------------------------------------------------------------|
| Oversize GPU segments |       0    |       0    |       0    |       0    |
|===========================================================================|

 CLEANUP:
 Allocated: 8.9GB
 Reserved: 11.3GB

 Cleanup completed.
 Allocated: 8.9GB
 Reserved: 11.3GB

 Cleanup Complete.
 Allocated: 8.9GB
 Reserved: 11.3GB

Steps:   0%|                               | 2/10400 [00:14<20:25:37,  7.07s/it, loss=0.00601, lr=2e-6, vram=5.7/9.5GB]
 Training completed, reloading SD Model.
 Allocated: 0.0GB
 Reserved: 9.5GB

Memory output: {}
 Restored system models.
 Allocated: 2.0GB
 Reserved: 9.5GB

Returning result: Training finished. Total lifetime steps: 802

Concept 0 class dir is G:\Program Files\stable-diffusion-webui\models\dreambooth\kobaryo_mdkb\classifiers_0
Starting Dreambooth training...
 Allocated 0.0/2.0GB
 Reserved: 0.0/2.2GB

Initializing dreambooth training...
Replace CrossAttention.forward to use default
Checking concept: {'max_steps': -1, 'instance_data_dir': 'D:\\Downloads\\Kobaryo_mdkb DreamBooth\\Train', 'class_data_dir': '', 'instance_prompt': 'by mdkb', 'class_prompt': '', 'save_sample_prompt': '(((extremely detailed unity 8K CG))), ((masterpiece)), (best quality), (an extremely delicate and beautiful), (intricate detail), (super delicate), illustration, hires, absurdres, highly detailed, rich detailed, beautiful detailed eyes, ((1girl)), ', 'save_sample_template': '', 'instance_token': '', 'class_token': '', 'num_class_images': 0, 'class_negative_prompt': '', 'class_guidance_scale': 7.5, 'class_infer_steps': 40, 'save_sample_negative_prompt': 'polar lowres, lowres, ((bad anatomy)), ((bad hands)), (bad body), bad proportions, gross proportions, text, error, missing fingers, (missing arms), (missing legs), extra digit, (extra fingers), fewer digits,\xa0(Mutated hands and fingers:1.5), one hand with more than 5 fingers, one hand with less than 5 fingers,  (extra limbs), (extra arms), (extra legs), (missing limbs), (missing legs), malformed limbs, fused fingers, (too many fingers), long neck, cross-eyed, mutated hands, cropped, poorly drawn hands, poorly drawn face, mutation, deformed, deformed legs, more than two knee, (bad feet), malformed feet, extra feet, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, fat, ugly, morbid, poorly drawn asymmetric eyes,\xa0bad shoes, fused shoes, poorly drawn shoes, bad gloves, poorly drawn gloves, fused gloves, bad hairs, poorly drawn hairs, fused hairs,bad eyes, fused eyes, poorly drawn eyes, extra eyes, [blurry], disfigured, bad mouth, fused mouth, poorly drawn mouth, dirty face, dirty teeth, humpbacked,\xa0', 'n_save_sample': 1, 'sample_seed': -1, 'save_guidance_scale': 10, 'save_infer_steps': 35}
Concept requires 0 images.
 Loaded model.
 Allocated: 0.0GB
 Reserved: 0.0GB

 Scheduler, EMA Loaded.
 Allocated: 3.8GB
 Reserved: 3.9GB

***** Running training *****
  Num examples = 83
  Num batches each epoch = 83
  Num Epochs = 126
  Instantaneous batch size per device = 1
  Total train batch size (w. parallel, distributed & accumulation) = 1
  Gradient Accumulation steps = 1
  Total optimization steps = 10400
  Actual steps: 10400
   Training settings: CPU: False Adam: True, Prec: fp16, Grad: True, TextTr: True EM: False, LR: 2e-06 LORA:False
 Allocated: 3.8GB
 Reserved: 3.9GB

Steps:   0%|                                            | 0/10400 [00:01<?, ?it/s, loss=0.317, lr=2e-6, vram=5.7/9.5GB] Step 0 completed.
 Allocated: 5.7GB
 Reserved: 9.5GB

Steps:   0%|                                 | 5/10400 [00:07<3:50:22,  1.33s/it, loss=0.298, lr=2e-6, vram=5.7/11.3GB] Step 5 completed.
 Allocated: 5.7GB
 Reserved: 11.3GB

Steps:   2%|▌                            | 198/10400 [03:57<3:22:00,  1.19s/it, loss=0.00814, lr=2e-6, vram=5.7/11.3GB]G:\Program Files\stable-diffusion-webui\venv\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py:151: FutureWarning: The configuration file of the unet has set the default `sample_size` to smaller than 64 which seems highly unlikely. If your checkpoint is a fine-tuned version of any of the following:
- CompVis/stable-diffusion-v1-4
- CompVis/stable-diffusion-v1-3
- CompVis/stable-diffusion-v1-2
- CompVis/stable-diffusion-v1-1
- runwayml/stable-diffusion-v1-5
- runwayml/stable-diffusion-inpainting
 you should change 'sample_size' to 64 in the configuration file. Please make sure to update the config accordingly as leaving `sample_size=32` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `unet/config.json` file
  deprecate("sample_size<64", "1.0.0", deprecation_message, standard_warn=False)
 Allocated 5.7/10.7GB
 Reserved: 6.6/11.3GB

Compiling checkpoint for kobaryo_mdkb...
Converting unet...
Converting vae...
Reshaping encoder.mid.attn_1.q.weight for SD format
Reshaping encoder.mid.attn_1.k.weight for SD format
Reshaping encoder.mid.attn_1.v.weight for SD format
Reshaping encoder.mid.attn_1.proj_out.weight for SD format
Reshaping decoder.mid.attn_1.q.weight for SD format
Reshaping decoder.mid.attn_1.k.weight for SD format
Reshaping decoder.mid.attn_1.v.weight for SD format
Reshaping decoder.mid.attn_1.proj_out.weight for SD format
Converting text encoder...
Converting text enc dict for V1 model.
Saving checkpoint to G:\Program Files\stable-diffusion-webui\models\Stable-diffusion\kobaryo_mdkb_1000.ckpt...
Generating samples: 100%|████████████████████████████████████████████████████████████████| 1/1 [00:13<00:00, 13.70s/it]
[*] Weights saved at G:\Program Files\stable-diffusion-webui\models\dreambooth\kobaryo_mdkb1/1 [00:13<00:00, 13.70s/it]
 Exception while training: local variable 'g_cuda' referenced before assignment
 Allocated: 5.7GB
 Reserved: 8.0GB

Traceback (most recent call last):
  File "G:\Program Files\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\train_dreambooth.py", line 1063, in main
    save_weights()
  File "G:\Program Files\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\train_dreambooth.py", line 942, in save_weights
    if g_cuda:
UnboundLocalError: local variable 'g_cuda' referenced before assignment
|===========================================================================|
|                  PyTorch CUDA memory summary, device ID 0                 |
|---------------------------------------------------------------------------|
|            CUDA OOMs: 1            |        cudaMalloc retries: 4         |
|===========================================================================|
|        Metric         | Cur Usage  | Peak Usage | Tot Alloc  | Tot Freed  |
|---------------------------------------------------------------------------|
| Allocated memory      |    5821 MB |    9773 MB |    9906 GB |    9900 GB |
|       from large pool |    5597 MB |    9484 MB |    9715 GB |    9709 GB |
|       from small pool |     223 MB |     293 MB |     191 GB |     191 GB |
|---------------------------------------------------------------------------|
| Active memory         |    5821 MB |    9773 MB |    9906 GB |    9900 GB |
|       from large pool |    5597 MB |    9484 MB |    9715 GB |    9709 GB |
|       from small pool |     223 MB |     293 MB |     191 GB |     191 GB |
|---------------------------------------------------------------------------|
| GPU reserved memory   |    8208 MB |    9966 MB |   58484 MB |   50276 MB |
|       from large pool |    7904 MB |    9660 MB |   57640 MB |   49736 MB |
|       from small pool |     304 MB |     306 MB |     844 MB |     540 MB |
|---------------------------------------------------------------------------|
| Non-releasable memory |     984 MB |    2212 MB |    6461 GB |    6460 GB |
|       from large pool |     944 MB |    2195 MB |    6261 GB |    6260 GB |
|       from small pool |      40 MB |      49 MB |     200 GB |     200 GB |
|---------------------------------------------------------------------------|
| Allocations           |    3708    |    4351    |    1894 K  |    1891 K  |
|       from large pool |     657    |     913    |     773 K  |     772 K  |
|       from small pool |    3051    |    3449    |    1121 K  |    1118 K  |
|---------------------------------------------------------------------------|
| Active allocs         |    3708    |    4351    |    1894 K  |    1891 K  |
|       from large pool |     657    |     913    |     773 K  |     772 K  |
|       from small pool |    3051    |    3449    |    1121 K  |    1118 K  |
|---------------------------------------------------------------------------|
| GPU reserved segments |     281    |     284    |     901    |     620    |
|       from large pool |     129    |     131    |     479    |     350    |
|       from small pool |     152    |     153    |     422    |     270    |
|---------------------------------------------------------------------------|
| Non-releasable allocs |     191    |     287    |    1011 K  |    1011 K  |
|       from large pool |      69    |     107    |     495 K  |     495 K  |
|       from small pool |     122    |     182    |     516 K  |     516 K  |
|---------------------------------------------------------------------------|
| Oversize allocations  |       0    |       0    |       0    |       0    |
|---------------------------------------------------------------------------|
| Oversize GPU segments |       0    |       0    |       0    |       0    |
|===========================================================================|

 CLEANUP:
 Allocated: 5.7GB
 Reserved: 8.0GB

 Cleanup completed.
 Allocated: 5.7GB
 Reserved: 6.6GB

 Cleanup Complete.
 Allocated: 5.7GB
 Reserved: 6.6GB

Steps:   2%|▌                            | 198/10400 [04:41<4:02:00,  1.42s/it, loss=0.00814, lr=2e-6, vram=5.7/11.3GB]
 Training completed, reloading SD Model.
 Allocated: 0.0GB
 Reserved: 3.8GB

Memory output: {}
 Restored system models.
 Allocated: 2.0GB
 Reserved: 3.8GB

Returning result: Training finished. Total lifetime steps: 1000

Environment

What OS? Windows 11 If Windows - WSL or native? Native What GPU are you using? RTX A2000 12GB Screenshots/Config db_config.json

{
    "adam_beta1": 0.9,
    "adam_beta2": 0.999,
    "adam_epsilon": 1e-08,
    "adam_weight_decay": 0.01,
    "attention": "default",
    "center_crop": false,
    "concepts_path": "",
    "gradient_accumulation_steps": 1,
    "gradient_checkpointing": true,
    "half_model": false,
    "hflip": true,
    "learning_rate": 2e-06,
    "lr_scheduler": "constant",
    "lr_warmup_steps": 500,
    "max_grad_norm": 1,
    "max_token_length": 75,
    "max_train_steps": 10400,
    "mixed_precision": "fp16",
    "model_dir": "G:\\Program Files\\stable-diffusion-webui\\models\\dreambooth\\kobaryo_mdkb",
    "model_name": "kobaryo_mdkb",
    "not_cache_latents": true,
    "num_train_epochs": 126,
    "pad_tokens": true,
    "pretrained_model_name_or_path": "G:\\Program Files\\stable-diffusion-webui\\models\\dreambooth\\kobaryo_mdkb\\working",
    "pretrained_vae_name_or_path": null,
    "prior_loss_weight": 1,
    "resolution": 512,
    "revision": 2000,
    "sample_batch_size": 1,
    "save_class_txt": true,
    "save_embedding_every": 500,
    "save_preview_every": 200,
    "save_use_global_counts": true,
    "save_use_epochs": false,
    "scale_lr": false,
    "src": "animefull-final-pruned.ckpt [925997e9]",
    "train_batch_size": 1,
    "train_text_encoder": true,
    "use_8bit_adam": true,
    "use_concepts": false,
    "use_cpu": false,
    "use_ema": false,
    "use_lora": false,
    "scheduler": "ddim",
    "v2": false,
    "has_ema": "False",
    "concepts_list": [
        {
            "max_steps": -1,
            "instance_data_dir": "D:\\Downloads\\Kobaryo_mdkb DreamBooth\\Train",
            "class_data_dir": "",
            "instance_prompt": "by mdkb",
            "class_prompt": "",
            "save_sample_prompt": "(((extremely detailed unity 8K CG))), ((masterpiece)), (best quality), (an extremely delicate and beautiful), (intricate detail), (super delicate), illustration, hires, absurdres, highly detailed, rich detailed, beautiful detailed eyes, ((1girl)), ((((by mdkb)))), ",
            "save_sample_template": "",
            "instance_token": "",
            "class_token": "",
            "num_class_images": 0,
            "class_negative_prompt": "",
            "class_guidance_scale": 7.5,
            "class_infer_steps": 40,
            "save_sample_negative_prompt": "polar lowres, lowres, ((bad anatomy)), ((bad hands)), (bad body), bad proportions, gross proportions, text, error, missing fingers, (missing arms), (missing legs), extra digit, (extra fingers), fewer digits,\u00a0(Mutated hands and fingers:1.5), one hand with more than 5 fingers, one hand with less than 5 fingers,  (extra limbs), (extra arms), (extra legs), (missing limbs), (missing legs), malformed limbs, fused fingers, (too many fingers), long neck, cross-eyed, mutated hands, cropped, poorly drawn hands, poorly drawn face, mutation, deformed, deformed legs, more than two knee, (bad feet), malformed feet, extra feet, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, fat, ugly, morbid, poorly drawn asymmetric eyes,\u00a0bad shoes, fused shoes, poorly drawn shoes, bad gloves, poorly drawn gloves, fused gloves, bad hairs, poorly drawn hairs, fused hairs,bad eyes, fused eyes, poorly drawn eyes, extra eyes, [blurry], disfigured, bad mouth, fused mouth, poorly drawn mouth, dirty face, dirty teeth, humpbacked,\u00a0",
            "n_save_sample": 1,
            "sample_seed": -1,
            "save_guidance_scale": 10,
            "save_infer_steps": 35
        }
    ],
    "lifetime_revision": 2000
}
i0ntempest commented 1 year ago

Oh... just noticed #475 fixes this.

i0ntempest commented 1 year ago

@d8ahazard I've added the required info, mind reopening?

Wyko commented 1 year ago 
try:
    if g_cuda:
        del g_cuda
except Exception:
    pass

At line 933 This fixes it, although in a quick and ugly way.

i0ntempest commented 1 year ago 
try:
    if g_cuda:
        del g_cuda
except Exception:
    pass

At line 933 This fixes it, although in a quick and ugly way.

I've done exactly that before I saw #475. Thx still.