search

zixaphir / Stable-Diffusion-Webui-Civitai-Helper

Stable Diffusion Webui Extension for Civitai, to manage your model much more easily.
175 stars 24 forks source link

[BUG] "ValueError: '' is not in list" on Aniverse v1.6 during Scan #67

Open bkosowski opened 6 months ago

bkosowski commented 6 months ago

The entire scanning process stops with an error when trying to process Aniverse v1.6 model.

Model can be downloaded from: https://civitai.com/models/107842/aniverse

The log:

Civitai Helper: Creating model info for: Aniverse_v16.safetensors
Civitai Helper: sha256: 96e558990766b5ec604cb4a0420fcf17ecfe9ce5277c8ce7ae26c25d1b20c05b
20:47:08-759086 DEBUG    Save: file="cache.json" json=2 bytes=17983 time=0.001
Civitai Helper: model action sha256: 96e558990766b5ec604cb4a0420fcf17ecfe9ce5277c8ce7ae26c25d1b20c05b
Civitai Helper: Request model info from civitai
Civitai Helper: Fetching Parent Model Information
Civitai Helper: Request model info from civitai: 107842
Civitai Helper: Write model civitai info to file: D:\AIArt\Models\Stable-diffusion\Animated\Aniverse_v16.civitai.info
Civitai Helper: Downloading model image.
Civitai Helper: Checking preview image for model: D:\AIArt\Models\Stable-diffusion\Animated\Aniverse_v16.safetensors
Civitai Helper: This image is NSFW: Soft
Traceback (most recent call last):
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\queueing.py", line 388, in call_prediction
    output = await route_utils.call_process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\route_utils.py", line 219, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\blocks.py", line 1437, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\blocks.py", line 1123, in call_function
    prediction = await utils.async_iteration(iterator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\utils.py", line 503, in async_iteration
    return await iterator.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\utils.py", line 496, in __anext__
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\utils.py", line 479, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\venv\Lib\site-packages\gradio\utils.py", line 629, in gen_wrapper
    yield from f(*args, **kwargs)
  File "D:\AIArt\automatic\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\model_action_civitai.py", line 181, in scan_model
    for _ in civitai.get_preview_image_by_model_path(
  File "D:\AIArt\automatic\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 492, in get_preview_image_by_model_path
    for result in verify_preview(
  File "D:\AIArt\automatic\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 401, in verify_preview
    if should_skip(nsfw_preview_threshold, image_rating):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIArt\automatic\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 360, in should_skip
    return order.index(image_rating) >= order.index(user_rating)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: '' is not in list
Osten commented 5 months ago

same but for another random embedding.

Traceback (most recent call last): File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict output = await app.get_blocks().process_api( File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api result = await self.call_function( File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1117, in call_function prediction = await utils.async_iteration(iterator) File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 350, in async_iteration return await iterator.anext() File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 343, in anext return await anyio.to_thread.run_sync( File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, args) File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 326, in run_sync_iterator_async return next(iterator) File "C:\Emil\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 695, in gen_wrapper yield from f(args, **kwargs) File "C:\Emil\AI\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\model_action_civitai.py", line 181, in scanmodel for in civitai.get_preview_image_by_model_path( File "C:\Emil\AI\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 492, in get_preview_image_by_model_path for result in verify_preview( File "C:\Emil\AI\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 401, in verify_preview if should_skip(nsfw_preview_threshold, image_rating): File "C:\Emil\AI\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 360, in should_skip return order.index(image_rating) >= order.index(user_rating) ValueError: '' is not in list

Osten commented 5 months ago

wait... is this just because "block NSFW level above" is null?

aba1t2x commented 4 months ago

我在扫描时总是提示错误，只要是nsfw的model，就报错，我已经Allow All了 Civitai Helper: GET Request failed with error code: 416: Range Not Satisfiable Traceback (most recent call last): File "D:\sd-webui-aki-v4.5\python\lib\site-packages\gradio\routes.py", line 488, in run_predict output = await app.get_blocks().process_api( File "D:\sd-webui-aki-v4.5\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api result = await self.call_function( File "D:\sd-webui-aki-v4.5\python\lib\site-packages\gradio\blocks.py", line 1117, in call_function prediction = await utils.async_iteration(iterator) File "D:\sd-webui-aki-v4.5\python\lib\site-packages\gradio\utils.py", line 350, in async_iteration return await iterator.anext() File "D:\sd-webui-aki-v4.5\python\lib\site-packages\gradio\utils.py", line 343, in anext return await anyio.to_thread.run_sync( File "D:\sd-webui-aki-v4.5\python\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "D:\sd-webui-aki-v4.5\python\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "D:\sd-webui-aki-v4.5\python\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "D:\sd-webui-aki-v4.5\python\lib\site-packages\gradio\utils.py", line 326, in run_sync_iterator_async return next(iterator) File "D:\sd-webui-aki-v4.5\python\lib\site-packages\gradio\utils.py", line 695, in gen_wrapper yield from f(args, **kwargs) File "D:\sd-webui-aki-v4.5\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\model_action_civitai.py", line 181, in scanmodel for in civitai.get_preview_image_by_model_path( File "D:\sd-webui-aki-v4.5\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 492, in get_preview_image_by_model_path for result in verify_preview( File "D:\sd-webui-aki-v4.5\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\civitai.py", line 414, in verify_preview for result in downloader.dl_file(img_url, file_path=path): File "D:\sd-webui-aki-v4.5\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\downloader.py", line 339, in dl_file yield from download_progress(url, file_path, total_size, headers, response) File "D:\sd-webui-aki-v4.5\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\downloader.py", line 142, in download_progress success, response_or_error = request_get( File "D:\sd-webui-aki-v4.5\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\downloader.py", line 68, in request_get response.raise_for_status() File "D:\sd-webui-aki-v4.5\python\lib\site-packages\requests\models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 416 Client Error: Range Not Satisfiable for url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/769e26dd-b40a-49ce-aa93-088ea9f1558b/width=1536/5874943.jpeg

SOELexicon commented 4 months ago

think this could do with skipping models that cant be found rather than just failing the entire run.

Civitai Helper: GET Request failed with error code: 404: Not Found Traceback (most recent call last): File "C:\AI\automatic\venv\lib\site-packages\gradio\queueing.py", line 388, in call_prediction output = await route_utils.call_process_api( File "C:\AI\automatic\venv\lib\site-packages\gradio\route_utils.py", line 219, in call_process_api output = await app.get_blocks().process_api( File "C:\AI\automatic\venv\lib\site-packages\gradio\blocks.py", line 1437, in process_api result = await self.call_function( File "C:\AI\automatic\venv\lib\site-packages\gradio\blocks.py", line 1123, in call_function prediction = await utils.async_iteration(iterator) File "C:\AI\automatic\venv\lib\site-packages\gradio\utils.py", line 503, in async_iteration return await iterator.anext() File "C:\AI\automatic\venv\lib\site-packages\gradio\utils.py", line 496, in anext return await anyio.to_thread.run_sync( File "C:\AI\automatic\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\AI\automatic\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "C:\AI\automatic\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, args) File "C:\AI\automatic\venv\lib\site-packages\gradio\utils.py", line 479, in run_sync_iterator_async return next(iterator) File "C:\AI\automatic\venv\lib\site-packages\gradio\utils.py", line 629, in gen_wrapper yield from f(args, **kwargs) File "C:\AI\automatic\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\model_action_civitai.py", line 162, in scan_model for result in scan_single_model(filepath, model_type, refetch_old, delay): File "C:\AI\automatic\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\model_action_civitai.py", line 93, in scan_single_model model_info = dummy_model_info(filepath, sha256_hash, model_type) File "C:\AI\automatic\extensions\Stable-Diffusion-Webui-Civitai-Helper\scripts\ch_lib\model_action_civitai.py", line 244, in dummy_model_info for tag in tag_frequency[trained_word].keys(): AttributeError: 'int' object has no attribute 'keys'

to get around these i have to rar any models/lora i have made before scanning otherwise the scan always fails

lhol commented 2 months ago

I had simmilar issues with models not found (404: Not found) because the default json created has no image tag and there is an exception in the code trying to parse that field stopping the scan. I have two different workarounds for that: Either append an empty image: [] array in the json created on models not found on civit.ai or (generally the better way) provide an try: ... error handler around single requests in case errors occur so that the scan can proceed with other models and does not stop at the same model over and over.

As for the error above: If the NSFW value is not a number, the parser fails. This should be handled by an try... or by a check, the question would be what assumption will be made with an invalid value in that field.