RupertAvery / DiffusionToolkit

Metadata-indexer and Viewer for AI-generated images
MIT License
709 stars 44 forks source link

Support metadata cache (via diskcache library) from A1111 sd-webui >= 1.9 #247

Open brendanhoar opened 3 weeks ago

brendanhoar commented 3 weeks ago

Is your feature request related to a problem? Please describe. Through A1111 version 1.8, the sd-webui model metadata was stored in a cache.json file. From 1.9 onward, it is now stored using the python diskcache library. I believe DT still relies on cache.json.

Describe the solution you'd like Read model metadata using the diskcache method.

Describe alternatives you've considered Keep an older version of A1111 around, and ensure every model is opened in that version at some point, pointing DT toward that particular cache.json for model hashes.

RupertAvery commented 3 weeks ago

Do you have more details about diskcache and how it's used in A1111? Not familiar with it. I really need to update my A1111 install.

brendanhoar commented 2 weeks ago

Here's the PR for the change in A1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15287 and the changes https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15287/files

And the docs: https://pypi.org/project/diskcache/

RupertAvery commented 2 weeks ago

It looks like diskcache uses sqlite as a backend, so going to that directly would probably be easier than replicating/porting the diskcache api

brendanhoar commented 2 weeks ago

Probably. You might run into some issues w/r/t database locks, however.

E.g. I've had DT crash itself repeatedly with database locking errors when internal timer threads try to access DT's own db during images scans. That case might be solvable with a catch/back-off/retry methodology.

B