search

philz1337x / clarity-upscaler

Clarity AI | AI Image Upscaler & Enhancer - free and open-source Magnific Alternative
https://ClarityAI.co
GNU Affero General Public License v3.0
3.93k stars 409 forks source link

Using custom sd model #43

Open anilsathyan7 opened 5 months ago

anilsathyan7 commented 5 months ago

Getting error in replicate demo (https://replicate.com/philz1337x/clarity-upscaler) and running same docker image locally, if we add sd_model url in options:-

File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/cog/server/worker.py", line 217, in _predict result = predict(**payload) File "/src/predict.py", line 463, in predict os.remove(path_to_custom_checkpoint) UnboundLocalError: local variable 'path_to_custom_checkpoint' referenced before assignment {"error": "local variable 'path_to_custom_checkpoint' referenced before assignment", "prediction_id": null, "logger": "cog.server.runner", "timestamp": "2024-06-29T10:56:39.592964Z", "severity": "INFO", "message": "prediction failed"}

custom_sd_model: "https://civitai.com/api/download/models/245598" Replicate logs:-

Running prediction
Upscaling with scale_factor:  2.0
Reusing loaded model juggernaut_reborn.safetensors to load epicrealism_naturalSinRC1VAE.safetensors
Loading weights [None] from /src/models/Stable-diffusion/epicrealism_naturalSinRC1VAE.safetensors
Loading VAE weights from commandline argument: models/VAE/vae-ft-mse-840000-ema-pruned.safetensors
Applying attention optimization: xformers... done.
Weights loaded in 1.1s (send model to cpu: 0.4s, apply weights to model: 0.2s, move model to device: 0.4s).
[Tiled Diffusion] upscaling image with 4x-UltraSharp...
[Tiled Diffusion] ControlNet found, support is enabled.
2024-06-29 11:08:19,333 - ControlNet - INFO - unit_separate = False, style_align = False
2024-06-29 11:08:19,333 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile
2024-06-29 11:08:19,356 - ControlNet - INFO - Using preprocessor: tile_resample
2024-06-29 11:08:19,356 - ControlNet - INFO - preprocessor resolution = 1536
2024-06-29 11:08:19,434 - ControlNet - INFO - ControlNet Hooked - Time = 0.1061091423034668
MultiDiffusion hooked into 'DPM++ 3M SDE Karras' sampler, Tile size: 144x112, Tile count: 4, Batch size: 4, Tile batches: 1 (ext: ContrlNet)
[Tiled VAE]: the input size is tiny and unnecessary to tile.
MultiDiffusion Sampling:   0%|          | 0/5 [00:00<?, ?it/s]
  0%|          | 0/7 [00:00<?, ?it/s]
Total progress:   0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:00<00:05,  1.01it/s]
Total progress:  29%|██▊       | 2/7 [00:00<00:01,  4.01it/s]
 29%|██▊       | 2/7 [00:01<00:03,  1.43it/s]
Total progress:  43%|████▎     | 3/7 [00:00<00:01,  2.84it/s]
 43%|████▎     | 3/7 [00:01<00:02,  1.65it/s]
Total progress:  57%|█████▋    | 4/7 [00:01<00:01,  2.46it/s]
 57%|█████▋    | 4/7 [00:02<00:01,  1.77it/s]
Total progress:  71%|███████▏  | 5/7 [00:01<00:00,  2.28it/s]
 71%|███████▏  | 5/7 [00:02<00:01,  1.85it/s]
Total progress:  86%|████████▌ | 6/7 [00:02<00:00,  2.18it/s]
 86%|████████▌ | 6/7 [00:03<00:00,  1.91it/s]
100%|██████████| 7/7 [00:03<00:00,  1.95it/s]
100%|██████████| 7/7 [00:03<00:00,  1.76it/s]
MultiDiffusion Sampling:   0%|          | 0/6 [00:26<?, ?it/s]
Total progress: 100%|██████████| 7/7 [00:02<00:00,  2.13it/s][Tiled VAE]: input_size: torch.Size([1, 4, 192, 192]), tile_size: 128, padding: 11
[Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 96x96, original tile size 128x128
[Tiled VAE]: Fast mode enabled, estimating group norm parameters on 128 x 128 image
[Tiled VAE]: Executing Decoder Task Queue:   0%|          | 0/492 [00:00<?, ?it/s]
[Tiled VAE]: Executing Decoder Task Queue:  25%|██▌       | 124/492 [00:00<00:00, 918.30it/s]
[Tiled VAE]: Executing Decoder Task Queue:  50%|█████     | 247/492 [00:00<00:00, 1028.06it/s]
[Tiled VAE]: Executing Decoder Task Queue:  75%|███████▌  | 370/492 [00:00<00:00, 1069.62it/s]
[Tiled VAE]: Executing Decoder Task Queue: 100%|██████████| 492/492 [00:00<00:00, 1114.69it/s]
[Tiled VAE]: Done in 1.201s, max VRAM alloc 5125.813 MB
Total progress: 100%|██████████| 7/7 [00:04<00:00,  2.13it/s]
Total progress: 100%|██████████| 7/7 [00:04<00:00,  1.55it/s]
Traceback (most recent call last):
File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/cog/server/worker.py", line 221, in _predict
result = predict(**payload)
File "/src/predict.py", line 574, in predict
os.remove(path_to_custom_checkpoint)
UnboundLocalError: local variable 'path_to_custom_checkpoint' referenced before assignment

{
  "completed_at": "2024-06-29T11:08:27.074199Z",
  "created_at": "2024-06-29T11:08:15.997000Z",
  "data_removed": false,
  "error": "local variable 'path_to_custom_checkpoint' referenced before assignment",
  "id": "s1f680kmznrj20cgceq8z4thcw",
  "input": {
    "seed": 1337,
    "image": "https://replicate.delivery/pbxt/KiDB5iqtTcxiTI17WASotG1Ei0TNJCztdU6J02pnMYAd8B1X/13_before-4.png",
    "prompt": "masterpiece, best quality, highres, <lora:more_details:0.5> <lora:SDXLrender_v2.0:1>",
    "dynamic": 6,
    "handfix": "disabled",
    "pattern": false,
    "sharpen": 0,
    "sd_model": "epicrealism_naturalSinRC1VAE.safetensors [84d76a0328]",
    "scheduler": "DPM++ 3M SDE Karras",
    "creativity": 0.35,
    "lora_links": "",
    "downscaling": false,
    "resemblance": 0.6,
    "scale_factor": 2,
    "tiling_width": 112,
    "output_format": "png",
    "tiling_height": 144,
    "custom_sd_model": "https://civitai.com/api/download/models/245598",
    "negative_prompt": "(worst quality, low quality, normal quality:2) JuggernautNegative-neg",
    "num_inference_steps": 18,
    "downscaling_resolution": 768
  },
  "logs": "Running prediction\nUpscaling with scale_factor:  2.0\nReusing loaded model juggernaut_reborn.safetensors to load epicrealism_naturalSinRC1VAE.safetensors\nLoading weights [None] from /src/models/Stable-diffusion/epicrealism_naturalSinRC1VAE.safetensors\nLoading VAE weights from commandline argument: models/VAE/vae-ft-mse-840000-ema-pruned.safetensors\nApplying attention optimization: xformers... done.\nWeights loaded in 1.1s (send model to cpu: 0.4s, apply weights to model: 0.2s, move model to device: 0.4s).\n[Tiled Diffusion] upscaling image with 4x-UltraSharp...\n[Tiled Diffusion] ControlNet found, support is enabled.\n2024-06-29 11:08:19,333 - ControlNet - \u001b[0;32mINFO\u001b[0m - unit_separate = False, style_align = False\n2024-06-29 11:08:19,333 - ControlNet - \u001b[0;32mINFO\u001b[0m - Loading model from cache: control_v11f1e_sd15_tile\n2024-06-29 11:08:19,356 - ControlNet - \u001b[0;32mINFO\u001b[0m - Using preprocessor: tile_resample\n2024-06-29 11:08:19,356 - ControlNet - \u001b[0;32mINFO\u001b[0m - preprocessor resolution = 1536\n2024-06-29 11:08:19,434 - ControlNet - \u001b[0;32mINFO\u001b[0m - ControlNet Hooked - Time = 0.1061091423034668\nMultiDiffusion hooked into 'DPM++ 3M SDE Karras' sampler, Tile size: 144x112, Tile count: 4, Batch size: 4, Tile batches: 1 (ext: ContrlNet)\n[Tiled VAE]: the input size is tiny and unnecessary to tile.\nMultiDiffusion Sampling:   0%|          | 0/5 [00:00<?, ?it/s]\n  0%|          | 0/7 [00:00<?, ?it/s]\u001b[A\u001b[A\nTotal progress:   0%|          | 0/7 [00:00<?, ?it/s]\u001b[A\n 14%|█▍        | 1/7 [00:00<00:05,  1.01it/s]\u001b[A\u001b[A\nTotal progress:  29%|██▊       | 2/7 [00:00<00:01,  4.01it/s]\u001b[A\n 29%|██▊       | 2/7 [00:01<00:03,  1.43it/s]\u001b[A\u001b[A\nTotal progress:  43%|████▎     | 3/7 [00:00<00:01,  2.84it/s]\u001b[A\n 43%|████▎     | 3/7 [00:01<00:02,  1.65it/s]\u001b[A\u001b[A\nTotal progress:  57%|█████▋    | 4/7 [00:01<00:01,  2.46it/s]\u001b[A\n 57%|█████▋    | 4/7 [00:02<00:01,  1.77it/s]\u001b[A\u001b[A\nTotal progress:  71%|███████▏  | 5/7 [00:01<00:00,  2.28it/s]\u001b[A\n 71%|███████▏  | 5/7 [00:02<00:01,  1.85it/s]\u001b[A\u001b[A\nTotal progress:  86%|████████▌ | 6/7 [00:02<00:00,  2.18it/s]\u001b[A\n 86%|████████▌ | 6/7 [00:03<00:00,  1.91it/s]\u001b[A\u001b[A\n100%|██████████| 7/7 [00:03<00:00,  1.95it/s]\u001b[A\u001b[A\n100%|██████████| 7/7 [00:03<00:00,  1.76it/s]\nMultiDiffusion Sampling:   0%|          | 0/6 [00:26<?, ?it/s]\nTotal progress: 100%|██████████| 7/7 [00:02<00:00,  2.13it/s]\u001b[A[Tiled VAE]: input_size: torch.Size([1, 4, 192, 192]), tile_size: 128, padding: 11\n[Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 96x96, original tile size 128x128\n[Tiled VAE]: Fast mode enabled, estimating group norm parameters on 128 x 128 image\n[Tiled VAE]: Executing Decoder Task Queue:   0%|          | 0/492 [00:00<?, ?it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue:  25%|██▌       | 124/492 [00:00<00:00, 918.30it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue:  50%|█████     | 247/492 [00:00<00:00, 1028.06it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue:  75%|███████▌  | 370/492 [00:00<00:00, 1069.62it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue: 100%|██████████| 492/492 [00:00<00:00, 1114.69it/s]\n[Tiled VAE]: Done in 1.201s, max VRAM alloc 5125.813 MB\nTotal progress: 100%|██████████| 7/7 [00:04<00:00,  2.13it/s]\u001b[A\nTotal progress: 100%|██████████| 7/7 [00:04<00:00,  1.55it/s]\nTraceback (most recent call last):\nFile \"/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/cog/server/worker.py\", line 221, in _predict\nresult = predict(**payload)\nFile \"/src/predict.py\", line 574, in predict\nos.remove(path_to_custom_checkpoint)\nUnboundLocalError: local variable 'path_to_custom_checkpoint' referenced before assignment",
  "metrics": {
    "predict_time": 11.065791937,
    "total_time": 11.077199
  },
  "output": null,
  "started_at": "2024-06-29T11:08:16.008408Z",
  "status": "failed",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/s1f680kmznrj20cgceq8z4thcw",
    "cancel": "https://api.replicate.com/v1/predictions/s1f680kmznrj20cgceq8z4thcw/cancel"
  },
  "version": "dfad41707589d68ecdccd1dfa600d55a208f9310748e44bfe35b4a6291453d5e"
}
michael-dm commented 4 months ago

I'm facing the exact same error.

StonedApeMedia commented 4 months ago

Still getting that error as well - Can anyone do the LORAS either? Don't get an error message using them but it doesn't seem to work at all.

alexrdzs commented 3 months ago

I'm also getting this error when trying to use _loralinks or _custom_sdmodel. I just can't figure it out or if I'm doing something wrong. Have you managed to make it work? I tried by pasting the downloadable link eg. https://civitai.com/api/download/models/40665 with no luck.