Problem with scanning

[paboum]

Getting this while trying to rescan my checkpoints. Version 203cd5a106fb143b51699e0a7d8e8c28a24442a6. Need to scan again and again until it succeeds.

Civitai Helper: GET Request failed with error code:
   503: Service Unavailable
Traceback (most recent call last):
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1117, in call_function
    prediction = await utils.async_iteration(iterator)
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/gradio/utils.py", line 350, in async_iteration
    return await iterator.__anext__()
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/gradio/utils.py", line 343, in __anext__
    return await anyio.to_thread.run_sync(
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/gradio/utils.py", line 326, in run_sync_iterator_async
    return next(iterator)
  File "/mnt/sdgui/venv/lib/python3.10/site-packages/gradio/utils.py", line 695, in gen_wrapper
    yield from f(*args, **kwargs)
  File "/mnt/sdgui/extensions/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model_action_civitai.py", line 162, in scan_model
    for result in scan_single_model(filepath, model_type, refetch_old, delay):
  File "/mnt/sdgui/extensions/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/model_action_civitai.py", line 90, in scan_single_model
    model_info = civitai.get_model_info_by_hash(sha256_hash)
  File "/mnt/sdgui/extensions/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/civitai.py", line 117, in get_model_info_by_hash
    content = append_parent_model_metadata(content)
  File "/mnt/sdgui/extensions/Stable-Diffusion-Webui-Civitai-Helper/scripts/ch_lib/civitai.py", line 86, in append_parent_model_metadata
    content["creator"] = parent_model.get("creator", "{}")
AttributeError: 'NoneType' object has no attribute 'get'

[zixaphir]

503 errors generally mean that Civitai is having issues. There's not really a whole lot I can do on this side except retry until it's working again

[paboum]

How about a user-configurable number of retries, delay or...

For each model, store the last successful check time. If it was less than, say, 15 minutes ago, don't retry updating this one as clearly it only wastes Civitai's resources. This way, when I rerun update, it will only retry the failed ones. User parameter "only download updates for elements older than ___ minutes" would be nice too.

[zixaphir]

Something akin to that is planned, but it will have to wait until after the holidays

[nl255]

503 errors generally mean that Civitai is having issues. There's not really a whole lot I can do on this side except retry until it's working again

I would suggest adding an option to skip it and continuing on with the next one rather than killing the entire scan simply because retrieving data for a single model failed. Logging the failed models to a text file and adding a "rescan failed models" button would be a nice extra.

[lhol]

If you scan a bigger library, 503 or 401 should lead to stop or as you said at least pause the scan, since civit ai is signaling an overall error or authentication issue. You could always press scan again (in case civit ai is having technical problems like a 503 it will stop nonetheless again). For other HTTP Error which more often signal problems with a single answer/dataset related to the model an automatic resume usually makes sense e.g. model not found in civit.ai.

[zixaphir]

Scanning should no longer fail due to network issues. If it is still failing, please let me know