The ‘Target directory for download’ stage is taking too long

[nomadoor]

For example, this is JoyTag, but every time I change the image, it takes 250 seconds to process, most of which is spent on ‘Target directory for download’ and ‘Fetching’.

got prompt
Target directory for download: D:\AI\ComfyUI_windows_portable\ComfyUI\models\LLavacheckpoints\files_for_joytagger
Fetching 6 files: 100%|██████████████████████████████████████████████████████████████████| 6/6 [01:21<00:00, 13.54s/it]
Model path: D:\AI\ComfyUI_windows_portable\ComfyUI\models\LLavacheckpoints\files_for_joytagger
Model path: D:\AI\ComfyUI_windows_portable\ComfyUI\models\LLavacheckpoints\files_for_joytagger
Prompt executed in 251.05 seconds

I don’t remember it taking this long before… I can’t think of any reason why.

[gokayfem]

well, it should not do that, did you check the inside the folder? are files there?

[nomadoor]

It seems that the files are properly in the specified folder. Of course, it’s an SSD.

It might not be related, but when I ran the program with the PC disconnected from the internet, the following error occurred.

Error occurred when executing Joytag:

[Errno 2] No such file or directory: 'C:\\Users\\---\\.cache\\huggingface\\hub\\models--fancyfeast--joytag\\snapshots\\6b7f16331a6ccf0fdce37d5a9564715f6e772b22\\config.json'

[gokayfem]

can you change the local_files_only=True in the snapshot_download function inside nodes/joytag.py

[nomadoor]

I tried it right away, but despite being connected to the internet, I started getting the same error as when the internet connection was cut off.

Error occurred when executing Joytag:

[Errno 2] No such file or directory: 'C:\\Users\\---\\.cache\\huggingface\\hub\\models--fancyfeast--joytag\\snapshots\\6b7f16331a6ccf0fdce37d5a9564715f6e772b22\\config.json'

[gokayfem]

can you change this

def download_joytag():
    # Ensure the correct behavior based on the existence of the local directory
    print(f"Target directory for download: {files_for_joytagger}")

    # Call snapshot_download with specified parameters
    path = snapshot_download(
        "fancyfeast/joytag",  # Example repo_id
        local_dir=files_for_joytagger,
        force_download=False,  # Set to True if you always want to download, regardless of local copy
        local_files_only=False,  # Set to False to allow downloading if not available locally
        local_dir_use_symlinks="auto"  # or set to True/False based on your symlink preference
    )
    print(f"Model path: {path}")
    return path

def download_joytag():
    # Ensure the correct behavior based on the existence of the local directory
    print(f"Target directory for download: {files_for_joytagger}")

    # Call snapshot_download with specified parameters
    path = files_for_joytagger
    print(f"Model path: {path}")
    return path

to this

[nomadoor]

It works perfectly! The speed is fantastic.

[gokayfem]

nice! if you are sure about you downloaded all of the files. you can change any snapshot_download function with files_for_... in any node that has this.

[nomadoor]

Thank you so much! However, I’m not very familiar with how git works. Will this code be overwritten in future updates?

I feel like we could avoid such issues by checking if files_for_joytagger exists before executing snapshot_download. What do you think?

[gokayfem]

yes it could be done, i have tried it messed it up but i will make it work soon.

[nomadoor]

There’s also a high possibility that the snapshot issue is being caused by my environment, so I’ll report back if I find out anything. Thank you😄

[gokayfem]

all i need to check is if the folder is empty or not, otherwise it should use the `files_for_... for each folder. i will do it soon.