feat: store model download paths in "path" key

kyteinsky commented 1 month ago

Models downloaded wrt models_to_fetch dict have the output path of the downloaded model files in the "path" key for each model.

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 0% with 7 lines in your changes missing coverage. Please review.

Project coverage is 94.65%. Comparing base (1bcd2b9) to head (ca87be0).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #274 +/- ## ========================================== - Coverage 94.69% 94.65% -0.04% ========================================== Files 47 47 Lines 5405 5407 +2 ========================================== Hits 5118 5118 - Misses 287 289 +2 ``` | [Files](https://app.codecov.io/gh/cloud-py-api/nc_py_api/pull/274?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloud-py-api) | Coverage Δ | | |---|---|---| | [nc\_py\_api/ex\_app/integration\_fastapi.py](https://app.codecov.io/gh/cloud-py-api/nc_py_api/pull/274?src=pr&el=tree&filepath=nc_py_api%2Fex_app%2Fintegration_fastapi.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloud-py-api#diff-bmNfcHlfYXBpL2V4X2FwcC9pbnRlZ3JhdGlvbl9mYXN0YXBpLnB5) | `24.51% <0.00%> (-0.33%)` | :arrow_down: |

bigcat88 commented 1 month ago

@kyteinsky should we merge this?

kyteinsky commented 1 month ago

hey, I was working on your (and Andrey's) proposed solution to have a key in the models dict itself. It works like this. Is it too complex?

class ModelConfig(dict):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def __setitem__(self, key, value):
        if key == "path":
            config["loader"]["hf_model_path"] = value
            service.load_config(config)
            save_config_file(config)

        super().__setitem__(key, value)

# download models if "model_name" key is present in the config
models_to_fetch = None
cache_dir = os.getenv("APP_PERSISTENT_STORAGE", "models/")
if "model_name" in config["loader"]:
    models_to_fetch = { config["loader"]["model_name"]: ModelConfig({ "cache_dir": cache_dir }) }

@asynccontextmanager
async def lifespan(_: FastAPI):
    set_handlers(
        fast_api_app=APP,
        enabled_handler=enabled_handler,
        models_to_fetch=models_to_fetch,
    )
    t = BackgroundProcessTask()
    t.start()
    yield

bigcat88 commented 1 month ago

I like it, it looks very nice :)

cloud-py-api / nc_py_api

feat: store model download paths in "path" key #274

Codecov Report