fix-civitai-helper-preview breaking bug

[Ainaemaet]

Hello,

I clone or checkout to the 'fix-civitai-helper-preview' branch, ComfyUI doesn't start up. Instead I get an infinite loading spinner (class="pi pi-spinner pi-spin text-3xl opacity-30").

At first, I assumed perhaps things were set to auto-scan on start, but I can't see anything happening on the backend, it just appears to be stuck.

I'm checking my files as it sits there t see if any .webp's are showing up that weren't there before, but it doesn't seem to be doing anything.

Is 'fix-civitai-helper-preview' the branch I should be on? Or is the main branch updated to the non-destructive proper scanning method as well?

[Ainaemaet]

Ok, right as I finished that comment - it finally finished loading the UI, however when I choose Model Manager settings, everything is exactly the same and it still warns that the process is destructive (which I believe it shouldn't be on this branch?).

At any rate, when clicking the migrate and continue on the warning, I get the same "[ComfyUI Model Manager] Migrating legacy information... [ComfyUI Model Manager] Download model info failed: 'images'" as before.

I don't think I'm doing anything wrong, it appears to be a problem in the migrator similar to before.

Sorry I can't be of more help

[hayden-fr]

I don't think you need to worry about migration anymore.

Your core requirement is that the preview image downloaded through a1111-civitai-helper cannot be displayed in ComfyUI-Model-Manager. This problem has been solved in the latest update. Switch to the branch main and pull.

[hayden-fr]

Or you can comment your questions and specific needs, and I will answer them for you. I will update the program if necessary.

[Ainaemaet]

@hayden-fr

Thank you for your attention to these needs. The extension is actually the a1111-civitai-browser-plus (https://github.com/BlafKing/sd-civitai-browser-plus).

It is a very powerful civ browser, enabling you to browse civ models, install, update, etc. and is also very fast. Perhaps there are some things in BlafKings repo relating to downloading and displaying previews and info that can help you improve this project? His project is worlds above the usability and functionality of civitai-helper; fully feature-rich, uncluttered and easy to use, etc.

I updated to main, still no previews showing - migrate worked better than before, but barely - this time instead of an immediate "Download model info failed: 'images'", it correctly migrated the description for one of them, and then failed.

I've decided to just go ahead with a full scan, but it is so slow compared to @cdb-boop's fork, which I'm assuming must have something to do with the fact that his automagically has much of my already downloaded info and previews available.

I am more than happy to provide you with any information from my own setup that you think might help you.

[Ainaemaet]

I do believe the BlafKing repo might be able to help with some things - it also seems to recognize all sorts of model data downloaded from other areas - and it is blazingly quick at downloading. Much faster than either this or cdboops fork.

Here are pictures of the options pages, just to give you an idea:

If you still have a auto1111 setup, try it out - it's amazing!!

[hayden-fr]

it correctly migrated the description for one of them, and then failed.

No failure message?

You can add the --verbose parameter at startup to get detailed

[hayden-fr]

I do believe the BlafKing repo might be able to help with some things - it also seems to recognize all sorts of model data downloaded from other areas - and it is blazingly quick at downloading. Much faster than either this or cdboops fork.

Thank you very much for your suggestions. It's truly useful to me. :)

[Ainaemaet]

it correctly migrated the description for one of them, and then failed.

No failure message?

You can add the --verbose parameter at startup to get detailed

Sounds good, I will try - thank you.

Glad to hear that repo might be helpful. I've always been kind of baffled that nothing similar had come along for ComfyUI yet. If you can give us similar functionality while keeping it relatively lightweight... Wow that would be amazing. 🦸

[Ainaemaet]

You may also want to check out the awesome ComfyUI Lora Sidebar plugin (https://github.com/Kinglord/ComfyUI_LoRA_Sidebar) as well.

I just gave it a try, and it not only displays super nicely - but is so blazingly fast in gathering info from Civitai (it also couldn't find my existing data, so I'm beginning to think that might be a problem with the fact I'm accessing ComfyUI from a networked PC using the --listen option? Not sure... 🤔).

Anyhow, perhaps you could consider collaborating or just checking out that repo as well to see what you can glean that may be of use?

[hayden-fr]

I do believe the BlafKing repo might be able to help with some things - it also seems to recognize all sorts of model data downloaded from other areas - and it is blazingly quick at downloading. Much faster than either this or cdboops fork.

I took a closer look at the repo a1111-civitai-browser-plus and found that it is indeed great, but it may not be what I want.

At present, the main problem of ComfyUI-Model-Manager is that it takes a lot of time to calculate the hash value of the model.

a1111-civitai-helper a1111-civitai-browser-plus has almost the same method to calculate hash values, and the method of ComfyUI-Model-Manager is consistent with the cdboops fork.

The principles of these two calculation methods are the same, there are slight differences in reading file chunks. There is not much difference in time consumption between the two methods in my end.

In details, a1111-civitai-browser-plus use a json file to store hash values ​​and use it as a cache file. If this file exists and has the sha256 attribute, use it directly.

If the user accidentally changes the sha256 value, the preview and information will not be updated correctly. Although this is very rare, it really bothers me.

I think it's great that a1111-civitai-browser-plus can selectively update by type when updating model information or preview, which gives me a good idea.

[Ainaemaet]

Every little bit helps. I'm still not getting things to display properly; on an attempted migration I get:

[ComfyUI Model Manager] Migrating legacy information...
[ComfyUI Model Manager] Migrate description from xl13AsmodeusSFWNSFW_v22BakedVAE.safetensors
[ComfyUI Model Manager] Migrate description from -Flux-/Flux.1-Dev/artsyDream_v4FP8.safetensors
[ComfyUI Model Manager] Migrate preview image from -Flux-/Flux.1-Dev/demonCORESFWNSFW_fluxV13_956597.safetensors
[ComfyUI Model Manager] Download model info failed: 'images'

The 'Download missing information or preview' works - in that it does scan through all my model folders (and then some), downloads and generates the required .webp's and .md's, but (almost) none of it displays in the UI.

I'm running ComfyUI over network using the '--listen' flag, not sure if that is causing the issue - but can't see why it would be?

Also, my model folder is symlinked from an ssd on the windows filesystem to Comfyui/models in WSL2 internally (via linux mount folder '/mnt') but there should be no issue there either, as linux recognizes it as just another folder and all of the other plugins, including cdboop's fork, the auto1111 ones (in auto oc), and every other model manager plugin for ComfyUI I've found picks them up just fine.

---

Ok, I think I've narrowed it down to a subdirectory naming-scheme issue.

So, I have most of my models organized in various subdir's, and the large majority of them start with either a hyphen '-' or tilde '~' to keep them easily accessible while using various nodes (it keeps them above those in the main /models/Stable-Diffusion or /models/Lora, etc.).

It appears your plugin doesn't account for this, as it is only those models that aren't in subdirectories to the main SD or Lora model folders, that are displaying properly.

I'm not sure if it's just a subdirectory issue, or if it is the fact my subdirs are named with tildes and hyphens that is the issue, because all of them are named with either one or the other.

Oh, some are named with a period at the start '.' as well, because those are usually used for 'hidden folder' designations in various AI art programs to keep certain models from being shown, unless they are checked off to show those ones in options somewhere.

I hope all that helps!!!!! :)

[hayden-fr]

Thanks for the information, I'll retest :)