search

containers / ai-lab-recipes

Examples for building and running LLM services and applications locally with Podman
Apache License 2.0
103 stars 106 forks source link

ilab wrapper script adjustments #686

Closed eranco74 closed 2 months ago

eranco74 commented 2 months ago

Background

RHEL AI ships with a script in /usr/bin called ilab which makes running ilab commands feel native even though they're actually running in a podman container

Issues

The abstraction becomes leaky once you start dealing with paths. The user thinks it's local paths, but it's actually paths inside the pod, and if the user is doing any action with a path that's not mounted inside the pod, files persisted to that path will not persist across ilab wrapper invocations

Examples:

  1. ilab config init outputs: Generating /root/.config/instructlab/config.yaml... Initialization completed successfully, you're ready to start using ilab. Enjoy! But: ls /root/.config/instructlab/config.yaml ls: cannot access '/root/.config/instructlab/config.yaml': Permission denied

  2. User provided paths e.g.: ilab config init --model-path... ilab model download --model-dir=...

The path may not be mounted to the host and the data is written to overlay fs and gone when the conatiner dies

Solution

Mount the user HOME direcotry and set the HOME inside the conainer This seems to resolve the abouve issues as long the user provided paths are nested under the user HOME direcotry