Mistral-Large-Instruct-2407-GGUF model does not work

[AdamKlob]

LM Studio 3.1 on Linux Ubuntu 24.04. Language model downloaded using LM Studio internal loader The error message is: Failed to send message Parse error. Expected else token. Closestatement !== Else.

[ryan-the-crayon]

Manually formatted prompt template:

{%- if messages[0]["role"] == "system" %}
  {%- set system_message = messages[0]["content"] %}
  {%- set loop_messages = messages[1:] %}
{%- else %}
  {%- set loop_messages = messages %}
{%- endif %}

{%- if not tools is defined %}
  {%- set tools = none %}
{%- endif %}

{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}

{#- This block checks for alternating user/assistant messages, skipping tool calling messages #}
{%- set ns = namespace() %}
{%- set ns.index = 0 %}

{%- for message in loop_messages %}
  {%- if not (message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
    {%- if (message["role"] == "user") != (ns.index % 2 == 0) %}
      {{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
    {%- endif %}

    {%- set ns.index = ns.index + 1 %}
  {%- endif %}
{%- endfor %}

{{- bos_token }}

{%- for message in loop_messages %}
  {%- if message["role"] == "user" %}
    {%- if tools is not none and (message == user_messages[-1]) %}
      {{- "[AVAILABLE_TOOLS] [" }}
      {%- for tool in tools %}
        {%- set tool = tool.function %}
        {{- '{"type": "function", "function": {' }}
        {%- for key, val in tool.items() if key != "return" %}
          {%- if val is string %}
            {{- '"' + key + '": "' + val + '"' }}
          {%- else %}
            {{- '"' + key + '": ' + val|tojson }}
          {%- endif %}
          {%- if not loop.last %}
            {{- ", " }}
          {%- endif %}
        {%- endfor %}
        {{- "}}" }}
        {%- if not loop.last %}
          {{- ", " }}
        {%- else %}
          {{- "]" }}
        {%- endif %}
      {%- endfor %}
      {{- "[/AVAILABLE_TOOLS]" }}
    {%- endif %}
    {%- if loop.last and system_message is defined %}
      {{- "[INST] " + system_message + "\n\n" + message["content"] + "[/INST]" }}
    {%- else %}
      {{- "[INST] " + message["content"] + "[/INST]" }}
    {%- endif %}
  {%- elif message.tool_calls is defined and message.tool_calls is not none %}
    {{- "[TOOL_CALLS] [" }}
    {%- for tool_call in message.tool_calls %}
      {%- set out = tool_call.function|tojson %}
      {{- out[:-1] }}
      {%- if not tool_call.id is defined or tool_call.id|length != 9 %}
        {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
      {%- endif %}
      {{- ', "id": "' + tool_call.id + '"}' }}
      {%- if not loop.last %}
        {{- ", " }}
      {%- else %}
        {{- "]" + eos_token }}
      {%- endif %}
    {%- endfor %}
  {%- elif message["role"] == "assistant" %}
    {{- " " + message["content"]|trim + eos_token}}
  {%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
    {%- if message.content is defined and message.content.content is defined %}
      {%- set content = message.content.content %}
    {%- else %}
      {%- set content = message.content %}
    {%- endif %}
    {{- '[TOOL_RESULTS] {"content": ' + content|string + ", " }}
    {%- if not message.tool_call_id is defined or message.tool_call_id|length != 9 %}
      {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
    {%- endif %}
    {{- '"call_id": "' + message.tool_call_id + '"}[/TOOL_RESULTS]' }}
  {%- else %}
    {{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
  {%- endif %}
{%- endfor %}

[ryan-the-crayon]

Problem is "{%- for key, val in tool.items() if key != "return" %}", which hf/jinja does not support. Updated to 0.3.0. Works now.

[RichSimon]

@ryan-the-crayon I'm using the Mac binary. What needs to be updated?

[yagil]

@RichSimon this will be included in the 0.3.3 update. If want to try out a beta when it's ready, please keep an eye on beta-releases-chat in our discord server. We'll share builds there in the next couple of days