Spike: rd-open-webui-docker-ext

mook-as commented 3 months ago

Look into rd-open-webui-docker-ext and see if we can get it into good enough shape to make it public.

mook-as commented 2 months ago

Looking into this, I think we will need to run a process on the host side (as it currently does); while it's easier on Windows, on Linux we'll need to make lima be able to expose devices somehow to get CUDA/ROCm to work. And on macOS, it seems unlike we will be able to expose Metal in a way that makes sense in a Linux VM.

We may wish to extend the extension format to specify a command to start a persistent host-side process in the background (and have RD manage restarting that process as needed). This would need to be per-platform, so likely under .host in metadata.json. Awkwardly, however, there are multiple files for Windows; we may need to figure out a good way to copy that (self-extracting archive and some sort of cleanup? Uninstall hook? Run the Linux version in WSL2 in a glibc-based container?).

gunamata commented 1 month ago

Here are my observations from trying out @mook-as 's fork so far:

Able to build, install (rdctl extention install ..) and use the extension on Windows. We need to confirm if ollama makes use of GPU available on the host machine.
On Linux, the extension installation fails.

extension.log

Loaded image: ghcr.io/rancher-sandbox/rancher-desktop/rdx-proxy:latest
2024-09-24T16:12:55.850Z: Failed to install extension matamagu/rd-open-webui-docker-ext, cleaning up: c [Error]: /opt/rancher-desktop/resources/resources/linux/bin/docker-compose exited with code 1
    at ChildProcess.<anonymous> (/opt/rancher-desktop/resources/app.asar/dist/app/background.js:2:193676)
    at ChildProcess.emit (node:events:519:28)
    at ChildProcess._handle.onexit (node:internal/child_process:294:12) {
  command: [
    '/opt/rancher-desktop/resources/resources/linux/bin/docker-compose',
    '--project-directory',
    '/home/guna/.local/share/rancher-desktop/extensions/bWF0YW1hZ3UvcmQtb3Blbi13ZWJ1aS1kb2NrZXItZXh0/compose',
    '--project-name',
    'rd-extension-matamagu_rd-open-webui-docker-ext',
    'up',
    '--quiet-pull',
    '--wait',
    '--remove-orphans'
  ],
  code: 1,
  [Symbol(child-process.command)]: '/opt/rancher-desktop/resources/resources/linux/bin/docker-compose --project-directory /home/guna/.local/share/rancher-desktop/extensions/bWF0YW1hZ3UvcmQtb3Blbi13ZWJ1aS1kb2NrZXItZXh0/compose --project-name rd-extension-matamagu_rd-open-webui-docker-ext up --quiet-pull --wait --remove-orphans'
}

moby.log

compose/compose.yaml, /home/guna/.local/share/rancher-desktop/extensions/bWF0YW1hZ3UvcmQtb3Blbi13ZWJ1aS1kb2NrZXItZXh0/compose/docker-compose.yaml"
time="2024-09-24T09:12:55-07:00" level=warning msg="Using /home/guna/.local/share/rancher-desktop/extensions/bWF0YW1hZ3UvcmQtb3Blbi13ZWJ1aS1kb2NrZXItZXh0/compose/compose.yaml"
 open-webui Pulling 
 searxng Pulling 
error getting credentials - err: exit status 1, out: `pass not initialized: exit status 1: Error: password store is empty. Try "pass init".`

However, the extension installation succeeds when tried with Rancher Desktop started via yarn dev.

On macOS, the extension installation succeeds but ollama is not installed or started so the open-webui app is not able to pull query models.

mook-as commented 1 month ago

Some of the feedback was that we're not uninstalling ollama when the extension is uninstalled. Unfortunately, there currently isn't an uninstall hook, so there's no way for us to actually do the uninstall.

We may need to add such a hook as an extension to the extension spec.

jandubois commented 2 weeks ago

The spike is done; any additional work will be tracked in separate issues, either here on in the extension repo.

rancher-sandbox / rancher-desktop

Spike: rd-open-webui-docker-ext #7321