Support for Differentiating Multiple Ollama Setups by Alias or Custom Name

devxpain commented 11 hours ago

Is your feature request related to a problem? Please describe. I have configured two ollama setups in the config.yaml file with different api_base values like so:

- type: openai-compatible
  name: ollama
  api_base: http://localhost:11434/v1
- type: openai-compatible
  name: ollama
  api_base: http://192.168.1.202:11434/v1

When using the .model <tab> command, I can see both ollama setups listed. However, when I try to use a model from the second ollama (e.g., qwen2.5-coder:14b), it always attempts to use the first ollama setup, even though I select the second ollama setup. The error I receive is:

Error: Failed to call chat-completions api

Caused by:
    model "qwen2.5-coder:14b" not found, try pulling it first (type: api_error)

This suggests that the system is always using the first ollama setup despite selecting the second one, likely because both setups share the same name ("ollama").

Describe the solution you'd like I would like the ability to differentiate between multiple ollama setups by either allowing aliases or providing a way to refactor the name field. This would allow each ollama setup to be used independently without the system defaulting to the first one.

Describe alternatives you've considered Currently, I have tried changing the name field to something unique, but this breaks the functionality because it seems like the name field is tied to the model type. I haven't found a way to alias the names without breaking the setup.

Additional context It would be helpful if there was a way to either:

Support aliases for name so that each ollama setup can have a unique alias without changing the underlying type.
Refactor the name field entirely to allow custom names for each ollama setup.

Let me know if you need further details. Thanks for considering this feature request!

sigoden commented 11 hours ago

Try changing the name to a new one (such as ollama2) and manually adding models.

- type: openai-compatible
  name: ollama2
  api_base: http://192.168.1.202:11434/v1
  models:
    - name: qwen2.5-coder
      max_input_tokens: 32768
      supports_function_calling: true

Each client configuration has a models field. For convenience, if you omit the models field, AIChat will use the default models defined in models.yaml.

devxpain commented 9 hours ago

Try changing the name to a new one (such as ollama2) and manually adding models.
- type: openai-compatible
  name: ollama2
  api_base: http://192.168.1.202:11434/v1
  models:
    - name: qwen2.5-coder
      max_input_tokens: 32768
      supports_function_calling: true
Each client configuration has a models field. For convenience, if you omit the models field, AIChat will use the default models defined in models.yaml.

Updated Suggestion: Add Optional `platform` Field

Thank you for considering my previous suggestion. To further improve flexibility while maintaining backward compatibility, I propose the following enhancement:

Proposed Improvement

Introduce an optional platform field in the configuration.
If the platform field is specified, it explicitly references the platform type.
If the platform field is omitted, the name field is used as the default reference for the platform.

Updated Example

Here’s how the configuration could look under the proposed system:

- type: openai-compatible
  name: ollama
  api_base: http://localhost:11434/v1

- type: openai-compatible
  name: ollama-remote
  platform: ollama
  api_base: http://192.168.1.202:11434/v1

In this example:

The first entry relies on the name field as the platform reference, maintaining backward compatibility.
The second entry explicitly specifies the platform field, making it clear that this configuration uses the ollama platform.

Benefits of This Approach

Backward Compatibility: Existing configurations remain unaffected, as the name field can still serve as the default reference.
Clarity and Flexibility: The optional platform field allows users to make the platform reference explicit when necessary, avoiding ambiguity.
Consistency: This aligns with the structure in models.yaml, where platforms are categorized explicitly.

Expected Behavior

If a platform is defined, it takes precedence and is used to identify the platform settings.
If platform is omitted, the name field is used as a fallback.

Why This Change?

This enhancement provides more flexibility and aligns with the intuitive understanding that the name is simply an identifier, while platform determines the underlying platform functionality.

Thank you for considering this suggestion! Let me know if you’d like further details or examples.

sigoden commented 9 hours ago

Introducing a new field complicates things, we will not accept the proposal.

To populate client models, Aichat first checks the models fields of the configuration.

- type: openai-compatible
  name: ...
  models:
    - name: qwen2.5-coder
      max_input_tokens: 32768
      supports_function_calling: true

If nothing is found, Aichat will check the client type, so the following configurations will work.

- type: claude
   name: claude1
   api_key: ...
- type: claude
   name: claude2
   api_key: ...

The openai-compatible client is speical, so we have to check the name.

- type: openai-compatible
  name: ollama

This project has been running for a long time, and only you have required this feature. Clearly, this is an edge case. We don't think it's worth introducing a new configuration item.

To resolve the issue, we will check the name using startsWith instead of using equals.

devxpain commented 9 hours ago

After testing the configuration:

  - type: openai-compatible
    name: ollama2
    api_base: http://localhost:11434/v1
    models:
      - name: nomic-embed-text
        type: embedding
        max_tokens_per_chunk: 8192
        default_chunk_size: 1000
        max_batch_size: 50

The models field are not appearing in the .model <tab> list.

sigoden commented 9 hours ago

Because .model only list chat models, but nomic-embed-text (type: embdding) is embedding models

sigoden / aichat