search

civitai / civitai-link-desktop

Manage your Civitai models for your Stable Diffusion Web UI models folder
16 stars 7 forks source link

Link downloads super slow compared to downloading from the site #29

Closed Brennok closed 3 months ago

Brennok commented 4 months ago

I have never used the link download before since I sort my models into subfolders using the base model and then additional sub-folders for things like styles, concepts, etc for embeddings and Loras.

Testing it now. Downloads are extremely slow. I don't know if this has always been the way, but testing two Loras I am 50 KB/s per Lora on a 1 gig fiber connection. Downloading on the site they were instant downloads.

ETA is currently 50 minutes for https://civitai.com/models/382919?modelVersionId=448897 as an example.

cmoody commented 4 months ago

@Brennok hmm weird. I just downloaded in a few seconds. Are you trying to download any other files at the same time? Although if its working downloading off the site it wouldnt be that. Let me noodle on that for a bit and Ill keep an eye out in case Im able to see similar.

Brennok commented 4 months ago

I wasn't downloading anything else through Link except the two Loras. The instant site downloader was at the same time so bandwidth wasn't an issue. Not a big deal if no one else is reporting, just wanted to mention in case. I will probably stick with downloading from the site for now just for organizing.

cmoody commented 4 months ago

Was just looking at the error that they sent over. For some reason Axios is erroring when making that preview image, unfortunately I am not really seeing it give a reason why. Running through a few ideas but at minimum I think I can keep it from crashing.

Ill add in a check to make sure it doesnt replace if theres an existing image.

Brennok commented 4 months ago

I will double check to see if any created successfully. I have API key stored so image rating shouldn't be an issue.

Brennok commented 3 months ago

I have narrowed down part of the issue. I think the closest cloudflarestorage assuming all files are duplicated across all storage sites is slow with SDXL and Pony size files. SD 1.5 is fast, but as soon as I pick anything larger than a 2gb checkpoint downloads cap around 1.5 MB/s or if I am lucky 2.5 MB/s. Tested across several computers on this network and same at each PC.

For example grabbing this random checkpoint, https://civitai.com/models/313958/faceless-void-mix-sdxl-pony, via link is about 2.5 MB/s, same with Chrome and Firefox. Running a VPN to Detroit over Miami and I jump to 30 MB/s in Firefox. and finished in a few minutes while Link still showed another hour or so. Even with the VPN and a fresh send to Link it still defaults to the slow download.

cmoody commented 3 months ago

@Brennok I believe there was a cloudflare issue routing some users around causing longer downloads. Not sure what the status of that issue is. However I did add some improvements to downloading which splits a file into multiple parts to download in parallel. I noticed a pretty significant improvement over download directly from the site.

If you wouldnt mind checking that out and also comparing to downloading directly off the site, that would be awesome.

Brennok commented 3 months ago

@cmoody SD 1.5 checkpoints in the 2gb size seem to download as fast if not slightly faster than via Firefox.

I am definitely seeing a faster download on Pony and SDXL checkpoints in the 6+gb filesize.