civitai / civitai-link-desktop

Manage your Civitai models for your Stable Diffusion Web UI models folder
16 stars 7 forks source link

Bug: Initial Setup Will Not Complete #21

Open bankenichi opened 3 months ago

bankenichi commented 3 months ago

After completing the initial setup, when clicking on Let's Go! the app will freeze for a moment and then do nothing. Repeating this process multiple times still does nothing. Closing and restarting the app yields the same result. Screenshot 2024-04-04 140645

ThermostatEnthusiast commented 3 months ago

I had the same problem, you have to let it register every model you have in your "models" folder. It took ages for me, no idea why it was using 600+ MB/s for such a long time for ~100GB in my "models"-folder

You can take a look at your Task-Manager and once the disk-usage drops to 0-ish MB/s you're good to go.

cmoody commented 3 months ago

Sorry for the late reply. @bankenichi I think this is because at start we hash each model so we can get information from the Civitai API about the model. As a result if you have a ton of files it can be slow and even crash. The latest version hopefully alleviates this and loads them in batches. Try it out and let me know.

bankenichi commented 3 months ago

The new installer still creates no icon in the task bar, meaning if you minimize it is gone, cant alt tab to it or get it back, the installer seems to just hang as well, doing nothing.

image

Brennok commented 3 months ago

Just as an update, I was able to complete install with 1.13.5. I did have to let it sit for a while before I could click Let's Go though. It did finally complete though and I definitely have a large number of files.

I did notice in settings though that Root Folder shows as blank now though. The other directories saved correctly.

cmoody commented 3 months ago

@Brennok thanks for the feedback. I added a fix in 1.13.5 that alleviated some pressure from a ton of files. Im working on a UI update that will also have a couple of other optimizations. Theres kind of two parts. One is that I think the large amount and size of files blocks the thread and is what causes it to be unresponsive. The other part Im thinking of adding a loading screen that shows the progress of hashing all of the files. Will keep yall updated.

Brennok commented 3 months ago

@cmoody One issue, which I can split into its own bug, none of the files installed that have been scanned so far, show as installed on civitai. I don't know if it needs to complete scanning before this works or not. It is still actively scanning, which I would probably suggest something to indicate this in the app. I almost put in a ticket for missing checkpoints, but they just now started showing up.

cmoody commented 3 months ago

Good catch! I think I forgot to emit the socket event when checking existing files. Right now I believe it only fires when downloaded or the site asks. It might work on refresh of the page but Ill get it added as part of initial load.

Brennok commented 3 months ago

@cmoody as far as I can tell, this finished indexing, but still not showing installed on civitai. I am also one of the people still having the connected cycle off and on on the website constantly so not sure if related, using latest firefox but tested in chrome also.

cmoody commented 3 months ago

as far as I can tell, this finished indexing, but still not showing installed on civitai.

Ive got a fix for that going out with the new UI, hopefully tomorrow.

The flapping of the socket connection is a strange one.

Brennok commented 3 months ago

@cmoody

I also tried all addons disabled with no luck so nothing tripping it that I can find.

Of course now that I say it I was just disconnected and not reconnected with the extension.

cmoody commented 3 months ago

The latest release(1.14.0) is out now. Hopefully fixes a handful of issues. Im going to leave this ticket open right now though because I still havent identified the socket issue. There was an issue with the socket server where it needed to be "woken up" but I dont think thats the issue you are having. I have rebuilt the socket server though and will be deploying that in the next day or so.

Brennok commented 3 months ago

@cmoody Do you want issues in this thread or seperate thread for each issue?

This pops up once. Unhandled Promise Rejection TypeError: Cannot read properties of undefined (reading 'hide') at C:/Users/me/AppData/Local/Programs/civitai-link/resources/app.asar/out/main/index.js:1368:23

Unhandled Promise Rejection Error: EMFILE: too many open files, open 'D:/Stable Diffusion/Automatic1111 Forge/webui/models/Stable-diffusion/SD Checkpoints/SD 1.5/halcyon_v20Bergamot.safetensors'

I did an upgrade and not a clean install, but I can do that if you prefer.

EDIT: I figured the below out. Turns out I had an old partial install that didn't get fully deleted in my Stable Diffusion folder which is where I keep all my models now. I use a separate main folder for my actual installs now.. Still getting the error, but nuked that old install so you can ignore the part about pulling from the wrong folders. This one is a bit odd, which I get constantly one for each model, since the location of the file is my SD install and not the main folder. My install was pointed to my models which I then symlink into the various installs. I can't get to Link to check the actual settings locations just due to the constant error popup so not sure if it defaulted to Automatic install. Unhandled Promise Rejection Error: EMFILE: too many open files, open 'D:/Stable Diffusion/Automatic1111 Forge/webui/models/Stable-diffusion/SD Checkpoints/SD 1.5/halcyon_v20Bergamot.safetensors'

cmoody commented 3 months ago

@Brennok Getting a patch out to prevent some of the noise and those popups.

Error: EMFILE: too many open files, open is one that I thought I had fixed with our updated batching. Ill need to investigate a little further. Its a bit of a tough one since I dont have that many models on my local. Im at an amateur 60 files at 48GB 🤣

Brennok commented 3 months ago

@cmoody I haven't even made my feature request to allow for a second folder for stored files for the files that are on my NAS due to not enough space on my NVME.

I think the too many files might have been that weird loop. I restarted without that folder there and I am not getting the EMFILE error. I do see the other error though at start up. Unhandled Promise Rejection TypeError: Cannot read properties of undefined (reading 'hide') at C:/Users/me/AppData/Local/Programs/civitai-link/resources/app.asar/out/main/index.js:1368:23

Also just got this error for the first time while replying. Unhandled Promise Rejection AggregateError at AxiosError.from (C:/Users/me/AppData/Local/Programs/civitai-link/resources/app.asar/node_modules/axios/dist/node/axios.cjs:836:14) at RedirectableRequest.handleRequestError (C:/Users/me/AppData/Local/Programs/civitai-link/resources/app.asar/node_modules/axios/dist/node/axios.cjs:3086:25) at RedirectableRequest.emit (node:events:514:28) at eventHandlers. (C:/Users/me/AppData/Local/Programs/civitai-link/resources/app.asar/node_modules/follow-redirects/index.js:38:24) at ClientRequest.emit (node:events:514:28) at TLSSocket.socketErrorListener (node:_http_client:495:9) at TLSSocket.emit (node:events:514:28) at emitErrorNT (node:internal/streams/destroy:151:8) at emitErrorCloseNT (node:internal/streams/destroy:116:3) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) at Axios.request (C:/Users/me/AppData/Local/Programs/civitai-link/resources/app.asar/node_modules/axios/dist/node/axios.cjs:3876:41) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async createPreviewImage (C:/Users/me/AppData/Local/Programs/civitai-link/resources/app.asar/out/main/index.js:461:20)

bankenichi commented 3 months ago

The latest version finally managed to complete the initial setup for me, however it is not reading all my resources, which somewhere around 4000 (loras, cpkt, tis, etc) and instead only detects around 1300ish. It's good progress tho, so I appreciate your efforts. Once setup and connected it seems like its having trouble downloading any resources from the site, instead creating a temp folder inside my loras root folder and creating a 0kb .safetensor for whatever i try to download, never actually finishing.

Edit just to add that it doesnt seem to recognize any kind of folder structure other than the root folder for each resource type.

cmoody commented 3 months ago

@bankenichi sorry for the late response. Can you check

By default, it writes logs to the following locations:

on Linux: ~/.config/{app name}/logs/main.log
on macOS: ~/Library/Logs/{app name}/main.log
on Windows: %USERPROFILE%\AppData\Roaming\{app name}\logs\main.log

and see if theres any information there?

For the difference in numbers, my guess is that they arent on Civitai. On first run we hash the files and then hit our API to see if we have that hash. There could be another issue which the logs might help with but thats just my initial guess. If a model was taken down from Civitai then this also could be the issue.

cmoody commented 2 months ago

Edit just to add that it doesnt seem to recognize any kind of folder structure other than the root folder for each resource type.

This probably explains the difference in file count. I think Ive got this all fixed in 1.15.1.

xuzhiping7 commented 2 months ago

i have the same problem , on version lastest 1.16.1.

the "let's go" btn click not happened anything ,just stuck in the page ...

i check the main.log , no error message :

[2024-05-14 00:46:39.434] [info] Starting App... [2024-05-14 00:46:39.491] [info] Checking for update [2024-05-14 00:46:39.492] [info] App ready [2024-05-14 00:46:42.653] [info] Update for version 1.16.1 is not available (latest version: 1.16.1, downgrade is disallowed).

cmoody commented 2 months ago

Hey everyone! Thanks for your patience on this. We just did a big overhaul to how we hash files and watch folders for models. Hoping this helps alleviate a lot of the slow startup issues and getting stuck on initial setup. Your app should automatically update to 1.18.0 but it might be worth reinstalling just so it doesnt try to start the download process in 1.16.1.

xuzhiping7 commented 2 months ago

i try the version 1.18 ,nothing change .

clicking on "Let's Go!" button the app then do nothing.

i guess may be the problem is about network error, but no any message can show that .and the main.log no any useful message.

Brennok commented 2 months ago

i try the version 1.18 ,nothing change .

clicking on "Let's Go!" button the app then do nothing.

i guess may be the problem is about network error, but no any message can show that .and the main.log no any useful message.

Early on I had a similar issue, if you never completed the initial setup you may want to do a complete uninstall, and reinstall. That fixed it for me when I had the problem at the start. I ended up using Revo Uninstaller to make sure I did a complete removal of Civitai Link since the uninstall seemed to leave data behind in the appdata folder on Windows.

xuzhiping7 commented 2 months ago

Revo Uninstaller

Thanks for reply , i try Revo Uninstaller as u say ,but not work .