Large Civitai.info files causing load issues with theming extensions

[Antivash]

Have you read document?

Yes

Have you checked console log window's msg?

Yes

Describe Issue

The images sections of civitai.info files is causing some to be unreasonably long, upwards of over 100k lines and is causing Nevysha's Cozy Nest extension and the Lobe theme to infinitely load the extra networks sections. Stripping that section out of the files resolves the issue.

Would it be possible to, either by default or as an option in the settings menu, have that section stripped out automatically? I don't currently see anything in the webui that uses that information, so ditching it doesnt really seem like a loss...

Screenshot for UI issue

Info File: https://i.imgur.com/Tmhsk3M.png Nevysha's Cozy Nest Theme Extra Networks: https://i.imgur.com/Ic1thse.png

Console log's msg or screenshot for function issue

Error in console; https://i.imgur.com/LuXsXCa.png

ERROR:    Exception in ASGI application
OverflowError: int too big to convert

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "K:\sdxl\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "K:\sdxl\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "K:\sdxl\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "K:\sdxl\venv\lib\site-packages\starlette\middleware\cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in __call__
    await responder(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in __call__
    await self.app(scope, receive, self.send_with_gzip)
  File "K:\sdxl\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "K:\sdxl\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "K:\sdxl\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "K:\sdxl\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "K:\sdxl\venv\lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
  File "K:\sdxl\venv\lib\site-packages\fastapi\routing.py", line 266, in app
    response = actual_response_class(content, **response_args)
  File "K:\sdxl\venv\lib\site-packages\starlette\responses.py", line 196, in __init__
    super().__init__(content, status_code, headers, media_type, background)
  File "K:\sdxl\venv\lib\site-packages\starlette\responses.py", line 55, in __init__
    self.body = self.render(content)
  File "K:\sdxl\venv\lib\site-packages\gradio\routes.py", line 80, in render
    return ORJSONResponse._render(content)
  File "K:\sdxl\venv\lib\site-packages\gradio\routes.py", line 73, in _render
    return orjson.dumps(
TypeError: Integer exceeds 64-bit range

[zixaphir]

Can you attach an info file with this issue? I want to examine potential causes and workshop solutions other than simply pruning that whole section

[Antivash]

Can you attach an info file with this issue? I want to examine potential causes and workshop solutions other than simply pruning that whole section

Sure. I kept all the originals after I striped what I was using. Tossed this one back in to verify that it was the image section causing it. Eimi_XL.civitai.info.zip

[zixaphir]

I'm not exactly sure what to do about this at the moment. For one, I'm not getting the same error messages you are, but the extra networks UI in Cozy Nest also wasn't loading at all for me. I was able to get around that by enabling "Deferred Loading" in Cozy Nest Modules.

But another thing is that this seems to be more of an issue with Cozy Nest than it does with my extension. The metadata files this script uses are mostly the same as they would be if you scraped the data from Civitai yourself, with just a few things moved around to provide additional data to the end user. The images section is entirely as-is from the API. I think this is more of an issue with Cozy Nest than it is with anything in Civitai Helper.

[zixaphir]

Slight addendum: Cozy Nest requires Butaixianran's Civitai Helper. I'm not aware of any incompatibilities that could be caused by using my version instead, but it might be worth switching to that version if you intend to follow up on this issue with the author of Cozy Nest.

[Antivash]

Slight addendum: Cozy Nest requires Butaixianran's Civitai Helper. I'm not aware of any incompatibilities that could be caused by using my version instead, but it might be worth switching to that version if you intend to follow up on this issue with the author of Cozy Nest.

Strange. Ive always been able to reproduce it with fresh installs, so I just assumed it was something easily done. But thanks for the heads up, Ill check out the version you linked and decide where to go from there.