Ability to Interact with Cached Models through an API

[Tostino]

Description

Currently, there is no direct way to interact with the cached models downloaded by PostgresML. For instance, when using HuggingFace models, they get stored in /var/lib/postgresql/.cache. Users might want to delete or even see what models have been downloaded.

I spoke with Lev on Discord and he mentioned that there is no API exposed to work with the cached models directly.

Suggested Feature

It would be helpful to provide an API for users to list, delete, or otherwise manage the models cached by PostgresML. This would give more control to the users over the storage and management of models without having to manually intervene at the file system level.

Use Case

1. Listing all downloaded models.
2. Deleting specific models that are no longer needed.
3. Potentially updating models.
4. Dealing with fine tuning models at some point

[montanalow]

I think we need to unify our 3 model concepts between PostgresML trained models, HuggingFace downloaded models, and SDK registered models as part of this effort. We're currently working on organizing the Dashboard, which can provide a UX to help organize/download/update/delete models, while providing additional visibility into their various stats, like disk/RAM usage as well as their key performance metrics, and usage.

There's a lot of deep functionality buried in various library APIs that would be good to surface in a UX, that also provides a reference SQL implementation for interacting with the pgml-extension to manage models across all the surfaces.

[montanalow]

770 is pretty cool, it's amazing what you can do in a Postgres extension, but I'm hesitant to expose a public API directly on the huggingface cache on disk. There are permission, replication and repeatability issues with the way we're using the cache on disk, so I'd rather it be short lived in it's current state. Ideally, we'd create a pgml.project and corresponding pgml.model with pgml.files for each huggingface transform task, similar to what pgml.tune does (except without the fine tuning).

What about adding pgml.load_model(project_name, task, **kwargs_for_automodel). Ultimately it'd be nice to add more structure for kwargs, but they are very dynamically handled in the various underlying libs. That'd also give end users the ability to group and deploy models how they see fit, allowing them to mix and match different model spaces for a single Project. The question is where all the huggingface model metadata get's stored. Perhaps the right place is the pgml.model.hyperparams. We'd also need a new "huggingface" algorithm, to handle running these correctly?