Open anilsathyan7 opened 4 weeks ago
Getting error in replicate demo (https://replicate.com/philz1337x/clarity-upscaler) and running same docker image locally, if we add sd_model url in options:-
File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/cog/server/worker.py", line 217, in _predict result = predict(**payload) File "/src/predict.py", line 463, in predict os.remove(path_to_custom_checkpoint) UnboundLocalError: local variable 'path_to_custom_checkpoint' referenced before assignment {"error": "local variable 'path_to_custom_checkpoint' referenced before assignment", "prediction_id": null, "logger": "cog.server.runner", "timestamp": "2024-06-29T10:56:39.592964Z", "severity": "INFO", "message": "prediction failed"}
custom_sd_model: "https://civitai.com/api/download/models/245598" Replicate logs:-
Running prediction Upscaling with scale_factor: 2.0 Reusing loaded model juggernaut_reborn.safetensors to load epicrealism_naturalSinRC1VAE.safetensors Loading weights [None] from /src/models/Stable-diffusion/epicrealism_naturalSinRC1VAE.safetensors Loading VAE weights from commandline argument: models/VAE/vae-ft-mse-840000-ema-pruned.safetensors Applying attention optimization: xformers... done. Weights loaded in 1.1s (send model to cpu: 0.4s, apply weights to model: 0.2s, move model to device: 0.4s). [Tiled Diffusion] upscaling image with 4x-UltraSharp... [Tiled Diffusion] ControlNet found, support is enabled. 2024-06-29 11:08:19,333 - ControlNet - [0;32mINFO[0m - unit_separate = False, style_align = False 2024-06-29 11:08:19,333 - ControlNet - [0;32mINFO[0m - Loading model from cache: control_v11f1e_sd15_tile 2024-06-29 11:08:19,356 - ControlNet - [0;32mINFO[0m - Using preprocessor: tile_resample 2024-06-29 11:08:19,356 - ControlNet - [0;32mINFO[0m - preprocessor resolution = 1536 2024-06-29 11:08:19,434 - ControlNet - [0;32mINFO[0m - ControlNet Hooked - Time = 0.1061091423034668 MultiDiffusion hooked into 'DPM++ 3M SDE Karras' sampler, Tile size: 144x112, Tile count: 4, Batch size: 4, Tile batches: 1 (ext: ContrlNet) [Tiled VAE]: the input size is tiny and unnecessary to tile. MultiDiffusion Sampling: 0%| | 0/5 [00:00<?, ?it/s] 0%| | 0/7 [00:00<?, ?it/s][A[A Total progress: 0%| | 0/7 [00:00<?, ?it/s][A 14%|█▍ | 1/7 [00:00<00:05, 1.01it/s][A[A Total progress: 29%|██▊ | 2/7 [00:00<00:01, 4.01it/s][A 29%|██▊ | 2/7 [00:01<00:03, 1.43it/s][A[A Total progress: 43%|████▎ | 3/7 [00:00<00:01, 2.84it/s][A 43%|████▎ | 3/7 [00:01<00:02, 1.65it/s][A[A Total progress: 57%|█████▋ | 4/7 [00:01<00:01, 2.46it/s][A 57%|█████▋ | 4/7 [00:02<00:01, 1.77it/s][A[A Total progress: 71%|███████▏ | 5/7 [00:01<00:00, 2.28it/s][A 71%|███████▏ | 5/7 [00:02<00:01, 1.85it/s][A[A Total progress: 86%|████████▌ | 6/7 [00:02<00:00, 2.18it/s][A 86%|████████▌ | 6/7 [00:03<00:00, 1.91it/s][A[A 100%|██████████| 7/7 [00:03<00:00, 1.95it/s][A[A 100%|██████████| 7/7 [00:03<00:00, 1.76it/s] MultiDiffusion Sampling: 0%| | 0/6 [00:26<?, ?it/s] Total progress: 100%|██████████| 7/7 [00:02<00:00, 2.13it/s][A[Tiled VAE]: input_size: torch.Size([1, 4, 192, 192]), tile_size: 128, padding: 11 [Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 96x96, original tile size 128x128 [Tiled VAE]: Fast mode enabled, estimating group norm parameters on 128 x 128 image [Tiled VAE]: Executing Decoder Task Queue: 0%| | 0/492 [00:00<?, ?it/s][A[A [Tiled VAE]: Executing Decoder Task Queue: 25%|██▌ | 124/492 [00:00<00:00, 918.30it/s][A[A [Tiled VAE]: Executing Decoder Task Queue: 50%|█████ | 247/492 [00:00<00:00, 1028.06it/s][A[A [Tiled VAE]: Executing Decoder Task Queue: 75%|███████▌ | 370/492 [00:00<00:00, 1069.62it/s][A[A [Tiled VAE]: Executing Decoder Task Queue: 100%|██████████| 492/492 [00:00<00:00, 1114.69it/s] [Tiled VAE]: Done in 1.201s, max VRAM alloc 5125.813 MB Total progress: 100%|██████████| 7/7 [00:04<00:00, 2.13it/s][A Total progress: 100%|██████████| 7/7 [00:04<00:00, 1.55it/s] Traceback (most recent call last): File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/cog/server/worker.py", line 221, in _predict result = predict(**payload) File "/src/predict.py", line 574, in predict os.remove(path_to_custom_checkpoint) UnboundLocalError: local variable 'path_to_custom_checkpoint' referenced before assignment { "completed_at": "2024-06-29T11:08:27.074199Z", "created_at": "2024-06-29T11:08:15.997000Z", "data_removed": false, "error": "local variable 'path_to_custom_checkpoint' referenced before assignment", "id": "s1f680kmznrj20cgceq8z4thcw", "input": { "seed": 1337, "image": "https://replicate.delivery/pbxt/KiDB5iqtTcxiTI17WASotG1Ei0TNJCztdU6J02pnMYAd8B1X/13_before-4.png", "prompt": "masterpiece, best quality, highres, <lora:more_details:0.5> <lora:SDXLrender_v2.0:1>", "dynamic": 6, "handfix": "disabled", "pattern": false, "sharpen": 0, "sd_model": "epicrealism_naturalSinRC1VAE.safetensors [84d76a0328]", "scheduler": "DPM++ 3M SDE Karras", "creativity": 0.35, "lora_links": "", "downscaling": false, "resemblance": 0.6, "scale_factor": 2, "tiling_width": 112, "output_format": "png", "tiling_height": 144, "custom_sd_model": "https://civitai.com/api/download/models/245598", "negative_prompt": "(worst quality, low quality, normal quality:2) JuggernautNegative-neg", "num_inference_steps": 18, "downscaling_resolution": 768 }, "logs": "Running prediction\nUpscaling with scale_factor: 2.0\nReusing loaded model juggernaut_reborn.safetensors to load epicrealism_naturalSinRC1VAE.safetensors\nLoading weights [None] from /src/models/Stable-diffusion/epicrealism_naturalSinRC1VAE.safetensors\nLoading VAE weights from commandline argument: models/VAE/vae-ft-mse-840000-ema-pruned.safetensors\nApplying attention optimization: xformers... done.\nWeights loaded in 1.1s (send model to cpu: 0.4s, apply weights to model: 0.2s, move model to device: 0.4s).\n[Tiled Diffusion] upscaling image with 4x-UltraSharp...\n[Tiled Diffusion] ControlNet found, support is enabled.\n2024-06-29 11:08:19,333 - ControlNet - \u001b[0;32mINFO\u001b[0m - unit_separate = False, style_align = False\n2024-06-29 11:08:19,333 - ControlNet - \u001b[0;32mINFO\u001b[0m - Loading model from cache: control_v11f1e_sd15_tile\n2024-06-29 11:08:19,356 - ControlNet - \u001b[0;32mINFO\u001b[0m - Using preprocessor: tile_resample\n2024-06-29 11:08:19,356 - ControlNet - \u001b[0;32mINFO\u001b[0m - preprocessor resolution = 1536\n2024-06-29 11:08:19,434 - ControlNet - \u001b[0;32mINFO\u001b[0m - ControlNet Hooked - Time = 0.1061091423034668\nMultiDiffusion hooked into 'DPM++ 3M SDE Karras' sampler, Tile size: 144x112, Tile count: 4, Batch size: 4, Tile batches: 1 (ext: ContrlNet)\n[Tiled VAE]: the input size is tiny and unnecessary to tile.\nMultiDiffusion Sampling: 0%| | 0/5 [00:00<?, ?it/s]\n 0%| | 0/7 [00:00<?, ?it/s]\u001b[A\u001b[A\nTotal progress: 0%| | 0/7 [00:00<?, ?it/s]\u001b[A\n 14%|█▍ | 1/7 [00:00<00:05, 1.01it/s]\u001b[A\u001b[A\nTotal progress: 29%|██▊ | 2/7 [00:00<00:01, 4.01it/s]\u001b[A\n 29%|██▊ | 2/7 [00:01<00:03, 1.43it/s]\u001b[A\u001b[A\nTotal progress: 43%|████▎ | 3/7 [00:00<00:01, 2.84it/s]\u001b[A\n 43%|████▎ | 3/7 [00:01<00:02, 1.65it/s]\u001b[A\u001b[A\nTotal progress: 57%|█████▋ | 4/7 [00:01<00:01, 2.46it/s]\u001b[A\n 57%|█████▋ | 4/7 [00:02<00:01, 1.77it/s]\u001b[A\u001b[A\nTotal progress: 71%|███████▏ | 5/7 [00:01<00:00, 2.28it/s]\u001b[A\n 71%|███████▏ | 5/7 [00:02<00:01, 1.85it/s]\u001b[A\u001b[A\nTotal progress: 86%|████████▌ | 6/7 [00:02<00:00, 2.18it/s]\u001b[A\n 86%|████████▌ | 6/7 [00:03<00:00, 1.91it/s]\u001b[A\u001b[A\n100%|██████████| 7/7 [00:03<00:00, 1.95it/s]\u001b[A\u001b[A\n100%|██████████| 7/7 [00:03<00:00, 1.76it/s]\nMultiDiffusion Sampling: 0%| | 0/6 [00:26<?, ?it/s]\nTotal progress: 100%|██████████| 7/7 [00:02<00:00, 2.13it/s]\u001b[A[Tiled VAE]: input_size: torch.Size([1, 4, 192, 192]), tile_size: 128, padding: 11\n[Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 96x96, original tile size 128x128\n[Tiled VAE]: Fast mode enabled, estimating group norm parameters on 128 x 128 image\n[Tiled VAE]: Executing Decoder Task Queue: 0%| | 0/492 [00:00<?, ?it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue: 25%|██▌ | 124/492 [00:00<00:00, 918.30it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue: 50%|█████ | 247/492 [00:00<00:00, 1028.06it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue: 75%|███████▌ | 370/492 [00:00<00:00, 1069.62it/s]\u001b[A\u001b[A\n[Tiled VAE]: Executing Decoder Task Queue: 100%|██████████| 492/492 [00:00<00:00, 1114.69it/s]\n[Tiled VAE]: Done in 1.201s, max VRAM alloc 5125.813 MB\nTotal progress: 100%|██████████| 7/7 [00:04<00:00, 2.13it/s]\u001b[A\nTotal progress: 100%|██████████| 7/7 [00:04<00:00, 1.55it/s]\nTraceback (most recent call last):\nFile \"/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/cog/server/worker.py\", line 221, in _predict\nresult = predict(**payload)\nFile \"/src/predict.py\", line 574, in predict\nos.remove(path_to_custom_checkpoint)\nUnboundLocalError: local variable 'path_to_custom_checkpoint' referenced before assignment", "metrics": { "predict_time": 11.065791937, "total_time": 11.077199 }, "output": null, "started_at": "2024-06-29T11:08:16.008408Z", "status": "failed", "urls": { "get": "https://api.replicate.com/v1/predictions/s1f680kmznrj20cgceq8z4thcw", "cancel": "https://api.replicate.com/v1/predictions/s1f680kmznrj20cgceq8z4thcw/cancel" }, "version": "dfad41707589d68ecdccd1dfa600d55a208f9310748e44bfe35b4a6291453d5e" }
I'm facing the exact same error.
Getting error in replicate demo (https://replicate.com/philz1337x/clarity-upscaler) and running same docker image locally, if we add sd_model url in options:-
File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/cog/server/worker.py", line 217, in _predict result = predict(**payload) File "/src/predict.py", line 463, in predict os.remove(path_to_custom_checkpoint) UnboundLocalError: local variable 'path_to_custom_checkpoint' referenced before assignment {"error": "local variable 'path_to_custom_checkpoint' referenced before assignment", "prediction_id": null, "logger": "cog.server.runner", "timestamp": "2024-06-29T10:56:39.592964Z", "severity": "INFO", "message": "prediction failed"}
custom_sd_model: "https://civitai.com/api/download/models/245598" Replicate logs:-