Issue with PASTE {target.upper()} URL OR GDRIVE PATH HERE option not working

[SilentCat096]

In section 1.2, download SDXL: def main(): global model_path, vae_path, LOAD_DIFFUSERS_MODEL

model_path, vae_path = None, None

required_sub_folders = [
    'scheduler',
    'text_encoder',
    'text_encoder_2',
    'tokenizer',
    'tokenizer_2',
    'unet',
    'vae',
]

download_targets = {
    "model": (SDXL_MODEL_URL, pretrained_model),
    "vae": (SDXL_VAE_URL, vae_dir),
}

total_ram = get_total_ram_gb()
gpu_name = get_gpu_name()

# Check hardware constraints
if total_ram < 13 and gpu_name in ["Tesla T4", "Tesla V100"]:
    print("Attempt to load diffusers model instead due to hardware constraints.")
    if not LOAD_DIFFUSERS_MODEL:
        LOAD_DIFFUSERS_MODEL = True

for target, (url, dst) in download_targets.items():
    if url and not url.startswith(f"PASTE {target.upper()} URL OR GDRIVE PATH HERE"):
        if target == "model" and LOAD_DIFFUSERS_MODEL:
            # Code for checking and handling diffusers model
            if 'huggingface.co' in url:
                match = re.search(r'huggingface\.co/([^/]+)/([^/]+)', SDXL_MODEL_URL)
                if match:
                    username = match.group(1)
                    model_name = match.group(2)
                    url = f"{username}/{model_name}"
            if all_folders_present(url, required_sub_folders):
                print(f"Diffusers model is loaded : {url}")
                model_path = url
            else:
                print("Repository doesn't exist or no diffusers model detected.")
                filepath = download(url, dst)  # Continue with the regular download
                model_path = filepath
        else:
            filepath = download(url, dst)

            if target == "model":
                model_path = filepath
            elif target == "vae":
                vae_path = filepath

        print()

if model_path:
    print(f"Selected model: {model_path}")

if vae_path:
    print(f"Selected VAE: {vae_path}")

You can see that this if: if url and not url.startswith(f"PASTE {target.upper()} URL OR GDRIVE PATH HERE"):

should else to this line:

else: filepath = download(url, dst)

But in code it is not, causing the "PASTE {target.upper()} URL OR GDRIVE PATH HERE" function to not work at all. Please double check the indentation.

[SilentCat096]

After double checking the code, I realized that it was caused by my overlook in the functionality. Upon detecting a "Tesla T4", "Tesla V100" instance, LOAD_DIFFUSERS_MODEL will be automatically set to true, going to the if target == "model" and LOAD_DIFFUSERS_MODEL: line, and the following error will be reported:

Attempt to load diffusers model instead due to hardware constraints.

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /usr/local/lib/python3.10/dist-packages/huggingface_hub/hf_file_system.py:101 in │ │ _repo_and_revision_exist │ │ │ │ 98 │ ) -> Tuple[bool, Optional[Exception]]: │ │ 99 │ │ if (repo_type, repo_id, revision) not in self._repo_and_revision_exists_cache: │ │ 100 │ │ │ try: │ │ ❱ 101 │ │ │ │ self._api.repo_info(repo_id, revision=revision, repo_type=repo_type) │ │ 102 │ │ │ except (RepositoryNotFoundError, HFValidationError) as e: │ │ 103 │ │ │ │ self._repo_and_revision_exists_cache[(repo_type, repo_id, revision)] = F │ │ 104 │ │ │ │ self._repo_and_revision_exists_cache[(repo_type, repo_id, None)] = False │ │ │ │ /usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py:110 in _inner_fn │ │ │ │ 107 │ │ │ kwargs.items(), # Kwargs values │ │ 108 │ │ ): │ │ 109 │ │ │ if arg_name in ["repo_id", "from_id", "to_id"]: │ │ ❱ 110 │ │ │ │ validate_repo_id(arg_value) │ │ 111 │ │ │ │ │ 112 │ │ │ elif arg_name == "token" and arg_value is not None: │ │ 113 │ │ │ │ has_token = True │ │ │ │ /usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py:164 in │ │ validate_repo_id │ │ │ │ 161 │ │ ) │ │ 162 │ │ │ 163 │ if not REPO_ID_REGEX.match(repoid): │ │ ❱ 164 │ │ raise HFValidationError( │ │ 165 │ │ │ "Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are" │ │ 166 │ │ │ " forbidden, '-' and '.' cannot start or end the name, max length is 96:" │ │ 167 │ │ │ f" '{repoid}'." │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'https:/'.

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ in <cell line: 171>:171 │ │ in main:147 │ │ in all_folders_present:88 │ │ │ │ /usr/local/lib/python3.10/dist-packages/huggingface_hub/hf_file_system.py:248 in ls │ │ │ │ 245 │ │ self, path: str, detail: bool = True, refresh: bool = False, revision: Optional[ │ │ 246 │ ) -> List[Union[str, Dict[str, Any]]]: │ │ 247 │ │ """List the contents of a directory.""" │ │ ❱ 248 │ │ resolved_path = self.resolve_path(path, revision=revision) │ │ 249 │ │ revision_in_path = "@" + safe_quote(resolved_path.revision) │ │ 250 │ │ has_revision_in_path = revision_in_path in path │ │ 251 │ │ path = resolved_path.unresolve() │ │ │ │ /usr/local/lib/python3.10/dist-packages/huggingface_hub/hf_file_system.py:165 in resolve_path │ │ │ │ 162 │ │ │ │ │ │ path_in_repo = path_in_repo_without_namespace │ │ 163 │ │ │ │ │ │ repo_and_revisionexist, = self._repo_and_revisionexist(repo │ │ 164 │ │ │ │ │ │ if not repo_and_revision_exist: │ │ ❱ 165 │ │ │ │ │ │ │ raise FileNotFoundError(path) from err │ │ 166 │ │ │ │ │ else: │ │ 167 │ │ │ │ │ │ raise FileNotFoundError(path) from err │ │ 168 │ │ else: │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ FileNotFoundError: https://civitai.com/api/download/models/128078

It should be easy to reproduce by setting 1.2's parameters to:

LOAD_DIFFUSERS_MODEL: False SDXL_MODEL_URL: https://civitai.com/api/download/models/128078

On a colab T4 instance. Without any explanation in the error message (or what load diffusers model is), it is confusing for the user. Would you please take a look into this, give more warnings about the option to download from custom URL? Thank you!

[TeeJayBaker]

Yeah, the download external model function does not work at all on SDXL for me either, the same civitai link style works fine on the SD1.X/2.X dreambooth colab instance.

Even if you do force bypass the resource check, it seems at the bucketing/latent caching stage it requires the model to be hosted on huggingface again anyway, so maybe more clear that civitai hosted models aren't supported