go-skynet / helm-charts

go-skynet helm chart repository
53 stars 38 forks source link

feat: add new parameter to pass models config files #46

Closed chnyda closed 2 months ago

chnyda commented 5 months ago

Adding modelsConfigs to be able to configure models with helm.

Why:

              for file in /models-configs/*; do
                filename=$(basename "$file")
                if [[ $filename != *.yaml ]]; then
                  cp -fL "$file" "/models/$filename.yaml"
                else
                  cp -fL "$file" "/models/$filename"
                fi
              done

Because I would like to make it usable with questions.yaml (used by Rancher and Taikun for example).

Example:

modelsConfigs:
  phi-2: |
    name: phi-2
    context_size: 2048
    f16: true
    #gpu_layers: 90
    mmap: true
    trimsuffix: 
    - "\n"
    parameters:
      model: huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
      temperature: 0.2
      top_k: 40
      top_p: 0.95
      seed: -1
    template:
      chat: &template |-
        Instruct: {{.Input}}
        Output:
      completion: *template

    usage: |
          To use this model, interact with the API (in another terminal) with curl for instance:
          curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
              "model": "phi-2",
              "messages": [{"role": "user", "content": "How are you doing?", "temperature": 0.1}]
          }'