Sanster / IOPaint

Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
https://www.iopaint.com/
Apache License 2.0
17.34k stars 1.75k forks source link

Allow inclusion of outside directories for more models #474

Open zono50 opened 2 months ago

zono50 commented 2 months ago

Is your feature request related to a problem? Please describe. I want to use models that are on a backup drive, and created a symlink, but the program does not find them. Wondering if there is a way to set a specific models directory, or concatenate a models directory so i have the models it downloads for usage, and then uses my models as well

Describe the solution you'd like A way to use backup directories to include outside models Additional context If i create a symlink to my models directory, i get the following output -

Traceback (most recent call last) ────────────────────────────────╮ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/typer_config/decorators.py:92 │ │ in wrapped │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/iopaint/cli.py:166 in start │ │ │ │ 163 │ │ │ 164 │ from iopaint.download import cli_download_model, scan_models │ │ 165 │ │ │ ❱ 166 │ scanned_models = scan_models() │ │ 167 │ if model not in [it.name for it in scanned_models]: │ │ 168 │ │ logger.info(f"{model} not found in {model_dir}, try to downloading") │ │ 169 │ │ cli_download_model(model) │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/iopaint/download.py:291 in │ │ scan_models │ │ │ │ 288 │ model_dir = os.getenv("XDG_CACHE_HOME", DEFAULT_MODEL_DIR) │ │ 289 │ available_models = [] │ │ 290 │ available_models.extend(scan_inpaint_models(model_dir)) │ │ ❱ 291 │ available_models.extend(scan_single_file_diffusion_models(model_dir)) │ │ 292 │ available_models.extend(scan_diffusers_models()) │ │ 293 │ available_models.extend(scan_converted_diffusers_models(model_dir)) │ │ 294 │ return available_models │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/iopaint/download.py:122 in │ │ scan_single_file_diffusion_models │ │ │ │ 119 │ │ model_abs_path = str(it.absolute()) │ │ 120 │ │ model_type = model_type_cache.get(it.name) │ │ 121 │ │ if model_type is None: │ │ ❱ 122 │ │ │ model_type = get_sd_model_type(model_abs_path) │ │ 123 │ │ model_type_cache[it.name] = model_type │ │ 124 │ │ res.append( │ │ 125 │ │ │ ModelInfo( │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/iopaint/download.py:59 in │ │ get_sd_model_type │ │ │ │ 56 │ │ from diffusers import StableDiffusionInpaintPipeline │ │ 57 │ │ │ │ 58 │ │ try: │ │ ❱ 59 │ │ │ StableDiffusionInpaintPipeline.from_single_file( │ │ 60 │ │ │ │ model_abs_path, │ │ 61 │ │ │ │ load_safety_checker=False, │ │ 62 │ │ │ │ num_in_channels=9, │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/huggingface_hub/utils/_validato │ │ rs.py:118 in _inner_fn │ │ │ │ 115 │ │ if check_use_auth_token: │ │ 116 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=ha │ │ 117 │ │ │ │ ❱ 118 │ │ return fn(*args, kwargs) │ │ 119 │ │ │ 120 │ return _inner_fn # type: ignore │ │ 121 │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/diffusers/loaders/single_file.p │ │ y:253 in from_single_file │ │ │ │ 250 │ │ │ if name in passed_class_obj: │ │ 251 │ │ │ │ init_kwargs[name] = passed_class_obj[name] │ │ 252 │ │ │ else: │ │ ❱ 253 │ │ │ │ components = build_sub_model_components( │ │ 254 │ │ │ │ │ init_kwargs, │ │ 255 │ │ │ │ │ class_name, │ │ 256 │ │ │ │ │ name, │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/diffusers/loaders/single_file.p │ │ y:58 in build_sub_model_components │ │ │ │ 55 │ │ │ 56 │ if component_name == "unet": │ │ 57 │ │ num_in_channels = kwargs.pop("num_in_channels", None) │ │ ❱ 58 │ │ unet_components = create_diffusers_unet_model_from_ldm( │ │ 59 │ │ │ pipeline_class_name, original_config, checkpoint, num_in_channels=num_in_cha │ │ 60 │ │ ) │ │ 61 │ │ return unet_components │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/diffusers/loaders/single_file_u │ │ tils.py:1166 in create_diffusers_unet_model_from_ldm │ │ │ │ 1163 │ unet_config["in_channels"] = num_in_channels │ │ 1164 │ unet_config["upcast_attention"] = upcast_attention │ │ 1165 │ │ │ ❱ 1166 │ diffusers_format_unet_checkpoint = convert_ldm_unet_checkpoint(checkpoint, unet_conf │ │ 1167 │ ctx = init_empty_weights if is_accelerate_available() else nullcontext │ │ 1168 │ with ctx(): │ │ 1169 │ │ unet = UNet2DConditionModel(unet_config) │ │ │ │ /home/zono50/anaconda3/envs/iopaint/lib/python3.10/site-packages/diffusers/loaders/single_file_u │ │ tils.py:673 in convert_ldm_unet_checkpoint │ │ │ │ 670 │ │ │ ) │ │ 671 │ │ │ 672 │ # Mid blocks │ │ ❱ 673 │ resnet_0 = middle_blocks[0] │ │ 674 │ attentions = middle_blocks[1] │ │ 675 │ resnet_1 = middle_blocks[2]

Not a programmer so not sure where i can go to add my directory, or what's the recommended method? I'm using latest version of Manjaro Linux

Tophness commented 1 month ago

Bumped at #494