search

charmbracelet / mods

AI on the command line
MIT License
2.86k stars 101 forks source link

Mods does not respect ollama base-url setting #311

Closed ActuallyAuggie closed 2 weeks ago

ActuallyAuggie commented 1 month ago

Mods seems to use the default ollama base-url: http://127.0.0.1:11434/api no matter what the line is set to.

Setup MX Linux 23 ahs, xfce4-term, bash

To Reproduce Change the line to anything, can be either a valid remote server or something nonsensical. The request will always go to the local machine.

Changing the base-url of, for instance, the openai config line does have the intended effect.

caustiq commented 4 weeks ago

Same here.

jahanson commented 3 weeks ago

This is the culprit: https://github.com/charmbracelet/mods/blob/3dc0a94ae6e9388de042a8ad4d260964ab71fdc5/mods.go#L315

should be occfg.BaseURL = api.BaseURL

mecattaf commented 2 weeks ago

My mods config is as follows:

default-model: llama3:8b
format-text:
  markdown: '{{ index .Config.FormatText "markdown" }}'
  json: '{{ index .Config.FormatText "json" }}'
roles:
  "default": []
  "summarize_git_diff":
    - https://raw.githubusercontent.com/danielmiessler/fabric/main/patterns/summarize_git_diff/system.md
format: false
role: "default"
raw: false
quiet: false
temp: 1.0
topp: 1.0
no-limit: false
word-wrap: 80
include-prompt-args: false
include-prompt: 0
max-retries: 5
fanciness: 10
status-text: Generating
max-input-chars: 12250
apis:
  ollama:
    base-url: http://localhost:11434/api
    models:
      "llama3:8b":
        aliases: ["llama3"]
        max-input-chars: 650000

When I try running a prompt through mods, I get the following in the ollama serve debug log

[GIN] 2024/08/25 - 12:26:32 | 404 |     415.694µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:32 | 404 |     111.536µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:32 | 404 |     180.034µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:33 | 404 |     109.918µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:35 | 404 |      112.41µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:45 | 404 |     165.582µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:45 | 404 |      230.76µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:45 | 404 |     171.141µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:46 | 404 |     151.101µs |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/08/25 - 12:26:48 | 404 |     171.085µs |       127.0.0.1 | POST     "/api/chat"

Will the associated PR solve my issue?