Can' t import models

[mairin]

Bug description

Hey :)

I have Podman Desktop 1.10.2 (flatpak) on Fedora 39 with Podman AI Lab v 1.1.2. (dropdown for bug version field only says 1.0.0)

I downloaded mixtral and wanted to import into Podman AI Lab. When I go to import it I get this error:

Error while importing models: ENOENT: no such file or directory, open '/home/duffy/.local/share/containers/podman-desktop/extensions-storage/redhat.ai-lab/user-catalog.json'

I tried making the directory and touching the file to have a blank file but that didn't work 😓

Operating system

fedora-release-workstation-39-36.noarch

Installation Method

from Podman-Desktop extension page

Version

1.0.0

Steps to reproduce

1. Get a Fedora 39 box
2. Install podman desktop from flathub
3. Install ai lab from the extensions screen in podman desktop
4. Download mixtral (and go have some lunch)
5. Come back, and go to models > import models in ai lab and see error

Relevant log output

No response

Additional context

No response

[mairin]

ok after cloning and grepping I found the README covers this a bit :)

To enhance the existing catalog, you can create a file located in the extension storage folder $HOME/.local/share/containers/podman-desktop/extensions-storage/redhat.ai-lab/user-catalog.json.

It must follow the same format as the default catalog in the sources of the extension.

Ideally though it would be good if the file didn't exist, for AI lab to create it for me.

In looking at creating my own I have the following questions:

• I'm creating the JSON needed by hand? something like?

{
  "models": [
    {
      "id": "hf.instructlab.granite-7b-lab-GGUF",
      "name": "instructlab/granite-7b-lab-GGUF",
      "description": "# InstructLab Granite 7B",
      "hw": "CPU",
      "registry": "Hugging Face",
      "license": "Apache-2.0",
      "url": "https://huggingface.co/instructlab/granite-7b-lab-GGUF/resolve/main/granite-7b-lab-Q4_K_M.gguf",
      "memory": 4080218931,
      "properties": {
        "chatFormat": "openchat"
      },
      "sha256": "6adeaad8c048b35ea54562c55e454cc32c63118a32c7b8152cf706b290611487"
    }
  ]
}

Is this right? So there isn't any metadata baked into the gguf to examine here? Would it make sense to have form fields on the model import screen to generate this for the user?

[mairin]

OK so I made a file in ~/.local/share/containers/podman-desktop/extensions-storage/redhat.ai-lab/user-catalog.json, with the following contents:

{
  "models": [
    {
 "id": "mixtral-8x7b-instruct-v0.1.Q4_0",
      "name": "mixtral-8x7b-instruct-v0.1.Q4_0",
      "description": "# Mixtral 8x7b instruct v0.1",
      "hw": "GPU",
      "registry": "Hugging Face",
      "license": "Apache-2.0",
      "url": "https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF",
      "memory": 2894000000,
      "properties": {
        "chatFormat": "openchat"
      },
      "sha256": "0c57465507f21bed4364fca37efd310bee92e25a4ce4f5678ef9b44e95830e4e"
    }
  ]
}

I tried to import the model again. It does not show up in the catalog nor the imported tab of the catalog. It does change the file on disk, however. Here is how the file looks after it is changed:

{
  "recipes": [],
  "models": [
    {
      "id": "mixtral-8x7b-instruct-v0.1.Q4_0",
      "name": "mixtral-8x7b-instruct-v0.1.Q4_0",
      "description": "# Mixtral 8x7b instruct v0.1",
      "hw": "GPU",
      "registry": "Hugging Face",
      "license": "Apache-2.0",
      "url": "https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF",
      "memory": 2894000000,
      "properties": {
        "chatFormat": "openchat"
      },
      "sha256": "0c57465507f21bed4364fca37efd310bee92e25a4ce4f5678ef9b44e95830e4e"
    },
    {
      "id": "/run/user/4154/doc/599852d6/mixtral-8x7b-instruct-v0.1.Q4_0.gguf",
      "name": "mixtral-8x7b-instruct-v0.1.Q4_0",
      "description": "Model imported from /run/user/4154/doc/599852d6/mixtral-8x7b-instruct-v0.1.Q4_0.gguf",
      "hw": "CPU",
      "file": {
        "path": "/run/user/4154/doc/599852d6",
        "file": "mixtral-8x7b-instruct-v0.1.Q4_0.gguf",
        "size": 26441533376,
        "creation": "2024-06-08T19:11:25.975Z"
      },
      "memory": 26441533376
    }
  ],
  "categories": []
}

So I went with that and trimmed the file down to just the imported one in the json... tried again. still doesn't show up 😭

[feloy]

Hey :)

I can reproduce the problem on Fedora 40 with Podman Desktop installed with Flatpak, and when there is no directory /.local/share/containers/podman-desktop/extensions-storage/redhat.ai-lab/ before to import a model (it worked well when the directory exists).

After creating the directory manually, and importing the model again, the model still didn't appear in the list. But by restarting the extension (or Podman Desktop), the imported model appeared as expected.

We will work on the fix to create the directory if it does not exist, and to see why the model is not correctly displayed before the extension is reloaded.

In the meantime, a workaround is to:

• create the directory manually (or start a recipe, which could create it for you)
• import the model
• restart the extension (or Podman Desktop)

[axel7083]

Error while importing models: ENOENT: no such file or directory, open '/home/duffy/.local/share/containers/podman-desktop/extensions-storage/redhat.ai-lab/user-catalog.json'

This issue has been fixed by https://github.com/containers/podman-desktop-extension-ai-lab/issues/1132

So I went with that and trimmed the file down to just the imported one in the json... tried again. still doesn't show up 😭

Yeah sadly there was another issue, which has been fixed by https://github.com/containers/podman-desktop-extension-ai-lab/issues/1132

In the next version of Podman AI Lab this process should be fixed :)

[axel7083]

CLosing as resolved in next release

[mairin]

Thanks for all of the help!!! 🙏