[Bug]: Title Generation does not work with llama3.1

[Phthisicus]

What happened?

I am running gemma2:9b and llama3.1:8b, and gemma2 does create titles. But llama3.1 does not.

My current config:

# Configuration version (required)
version: 1.1.5

# Cache settings: Set to true to enable caching
cache: true

endpoints:
  custom:
    - name: "Ollama"
      apiKey: "ollama"
      baseURL: "http://x.x.x.x:11434/v1/"
      models:
        default: [
          "gemma2:9b",
          "llama3.1:8b"
          ]
      fetch: false # fetching list of models is not supported
      titleConvo: true
      titleModel: "current_model"
      summarize: true
      summaryModel: "current_model"
      modelDisplayLabel: "Ollama"

Steps to Reproduce

Switching from gemma2 to llama3.1 in the interface is enough to reproduce the problem.

What browsers are you seeing the problem on?

Chrome

Relevant log output

No response

Screenshots

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[danny-avila]

Any server logs when this happens?

[sparkeh]

I have the same issue. Here is some of the debug logs I gathered recently.

I asked llama3.1 question "How many days in a week?"

2024-08-13T11:28:27.407Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "http://10.1.1.11:11434/v1",
    modelOptions.model: "llama3.1:latest",
    modelOptions.temperature: 1,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.max_tokens: undefined,
    modelOptions.user: "6658527ad5f389f3fb89cfd2",
    modelOptions.stream: true,
    // 1 message(s)
    modelOptions.messages: [{"role":"user","content":"How many days in a week?"}],
}
2024-08-13T11:28:27.408Z debug: Making request to http://10.1.1.11:11434/v1/chat/completions
2024-08-13T11:28:27.410Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo

Response:

2024-08-13T11:28:29.332Z debug: [OpenAIClient] chatCompletion response
{
  object: "chat.completion",
  id: "chatcmpl-47",
  // 1 choice(s)
  choices: [{"message":{"role":"assistant","content":"There are 7 days a week:\n\n1. Monday\n2. Tuesday\n3. Wedn... [truncated]],
  created: 1723548509,
  model: "llama3.1:latest",
  system_fingerprint: "fp_ollama",
}
2024-08-13T11:28:29.333Z debug: [spendTokens] conversationId: 7880f12e-3896-4548-ad83-9e26346bf984 | Context: message | Token usage:
{
  promptTokens: 14,
  completionTokens: 35,
}
2024-08-13T11:28:29.339Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2024-08-13T11:28:29.346Z debug: [AskController] Request closed

Chat topic request:

2024-08-13T11:28:29.351Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "http://10.1.1.11:11434/v1",
    modelOptions.model: "llama3.1:latest",
    modelOptions.temperature: 0.2,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.max_tokens: 16,
    modelOptions.user: "6658527ad5f389f3fb89cfd2",
    // 1 message(s)
    modelOptions.messages: [{"role":"system","content":"Please generate a concise, 5-word-or-less title for the conversation, us... [truncated]],
}
2024-08-13T11:28:29.352Z debug: Making request to http://10.1.1.11:11434/v1/chat/completions

Chat topic response:

2024-08-13T11:28:30.078Z debug: [OpenAIClient] chatCompletion response
{
  id: "chatcmpl-807",
  object: "chat.completion",
  created: 1723548510,
  model: "llama3.1:latest",
  system_fingerprint: "fp_ollama",
  // 1 choice(s)
  choices: [{"index":0,"message":{"role":"assistant","content":""},"finish_reason":"stop"}],
    usage.prompt_tokens: 110,
    usage.completion_tokens: 1,
    usage.total_tokens: 111,
}
2024-08-13T11:28:30.079Z debug: [spendTokens] conversationId: undefined | Context: title | Token usage:
{
  promptTokens: 108,
  completionTokens: 0,
}
2024-08-13T11:28:30.079Z debug: [OpenAIClient] Convo Title:
2024-08-13T11:28:30.079Z debug: [saveConvo] api/server/services/Endpoints/openAI/addTitle.js

I note that the content is blank as well as Convo Title

Testing with Gemma2 I would observe some content: EG:

2024-08-13T11:30:03.002Z debug: [OpenAIClient] chatCompletion response
{
  id: "chatcmpl-121",
  object: "chat.completion",
  created: 1723548603,
  model: "gemma2:latest",
  system_fingerprint: "fp_ollama",
  // 1 choice(s)
  choices: [{"index":0,"message":{"role":"assistant","content":"Seven Days In A Week  \n"},"finish_reason":"stop... [truncated]],
    usage.prompt_tokens: 106,
    usage.completion_tokens: 8,
    usage.total_tokens: 114,
}
2024-08-13T11:30:03.002Z debug: [spendTokens] conversationId: undefined | Context: title | Token usage:
{
  promptTokens: 98,
  completionTokens: 5,
}
2024-08-13T11:30:03.003Z debug: [OpenAIClient] Convo Title: Seven Days In A Week
2024-08-13T11:30:03.003Z debug: [saveConvo] api/server/services/Endpoints/openAI/addTitle.js

[danny-avila]

This can be fixed with configuration, setting titleMessageRole to user for Ollama:

https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/custom_endpoint#titlemessagerole

But I've submitted a PR to address this in case a role is never specified

https://github.com/danny-avila/LibreChat/pull/3663