lmstudio-ai / lms

👾 LM Studio CLI
https://lms.dev
MIT License
1.69k stars 138 forks source link

Can this be used with "Ollama" to share models? If so, how can it be modified? #72

Open Willy-Shenn opened 2 months ago

Willy-Shenn commented 2 months ago

I am currently using two UI systems, but they cannot share models (possibly due to differences in how the models are identified and created). Even after modifying the environment variables, both UIs cannot use models from the same path. Is there anyone who can guide me on how to modify the two UIs so they can use models from the same path? I would be very grateful.

ink-splatters commented 2 months ago

Option 1

  1. Install gollama. It supports some cli options but defaults should be fine in the most cases
  2. Make sure lms models dir exists, the snippet below assumes its default location:
mkdir -p ~/.cache/lm-studio/models
  1. in gollama TUI use l/ L shortcuts to (re)link particular / all models

Option 2

  1. install jq
  2. Use this script to (re)link all models:
#!/usr/bin/env bash
# NOTE: this script should be properly credited as I only slightly modified it, but unfortunately I don't know the author's name anymore.

OLLAMA_MODELS_DIR="${OLLAMA_MODELS_DIR:-$HOME/.ollama/models}"
LMSTUDIO_MODELS_DIR="${LMSTUDIO_MODELS_DIR:-$HOME/.cache/lm-studio/models}"

OLLAMA_MANIFEST_DIR=$OLLAMA_MODELS_DIR/manifests/registry.ollama.ai
OLLAMA_BLOBS_DIR=$OLLAMA_MODELS_DIR/blobs

mkdir -p "$LMSTUDIO_MODELS_DIR"
find "$LMSTUDIO_MODELS_DIR" -type l -exec rm {} +

find "$OLLAMA_MANIFEST_DIR" -mindepth 3 -maxdepth 3 -type f | while IFS= read -r file; do
    user=$(basename "$(dirname "$(dirname "$file")")" | sed 's/^registry\.ollama\.ai/ollama/')
    model=$(basename "$(dirname "$file")")
    tag=$(basename "$file")

    digest=$(jq -r '.layers[] | select(.mediaType == "application/vnd.ollama.image.model") | .digest' "$file")
    digest="${digest//:/-}"
    mkdir -p "$LMSTUDIO_MODELS_DIR/$user/$model/"
    ln -s "$OLLAMA_BLOBS_DIR/$digest" "$LMSTUDIO_MODELS_DIR/$user/$model/$model-$tag.gguf"

    echo "$user - $model:$tag"
done