search

containers / podman-desktop-extension-ai-lab

Work with LLMs on a local environment using containers
https://podman-desktop.io/extensions/ai-lab
Apache License 2.0
182 stars 41 forks source link

Podman Desktop AI Lab model list can be configured to other sources #1847

Open amfred opened 1 month ago

amfred commented 1 month ago

Is your feature request related to a problem? Please describe

Podman AI Lab offers a curated list of models that users can easily grab and run locally. Users may want to configure Podman AI Lab to reference their organization's own curated list or other trusted source of compatible models. The primary need for this is in organizations where developers should NOT be pulling arbitrary models from Hugging Face.

If a company has a vetted list of approved models, they should be able to make it easy for their developers to access and use those models. One way of accomplishing that would be to make that process easy within Podman AI Lab. Depending on customer needs, perhaps this would replace the default model list, to help keep devs on an approved path and prevent wasting time evaluating models that their company does not trust.

Describe the solution you'd like

As a Red Hat customer, I should be able to tell my developers how to configure Podman AI Lab, and when they follow those configuration steps, they should only see the list of models that I want my developers to use in their applications.

As an example, a RHDH customer could configure Podman AI Lab to show models that are listed in the AI Catalog in Developer Hub (RHDP-1014).

Or pull data from OpenShift AI's model registry (there are likely to be multiple model registries in each company, so really, it's pulling models from a list of registries).

This could also be a config file that contains the map of visible models and their download locations.

Some important registries that are available today include: Hugging Face, IBM Watson, Artifactory, Local (on my laptop). Instruct Lab and RHOAI in the future.

Describe alternatives you've considered

No response

Additional context

Related internal Jira ticket: RHDP-1059

deboer-tim commented 1 month ago

The catalog is already replaceable (https://github.com/containers/podman-desktop-extension-ai-lab/blob/main/PACKAGING-GUIDE.md) and there's an existing issue for other model registries (#476).