Errors when running example script

[DePasqualeOrg]

When running the example script from the readme, I get the following error:

ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating

[DePasqualeOrg]

I was able to resolve this by manually setting the chat template:

llava_1_5_chat_template = '''{% for message in messages %}
  {% if message.role == 'system' %}
    {{ message.content.strip() }}
    {{ '\n' }}

  {% elif message.role == 'user' %}
    {% if message.content is string %}
      USER: {{ message.content.strip() }}

    {% elif message.content is iterable %}
      USER: 

      {% for content in message.content %}

        {% if content.type == 'image_url' and content.image_url is mapping and content.image_url.url is string%}
          {{ content.image_url.url.strip() + ' ' }}
        {% endif %}

      {% endfor %}

      {% for content in message.content %}

        {% if content.type == 'text' %}
          {{ content.text.strip() }}
        {% endif %}

      {% endfor %}

    {% endif %}

    {{ '\n' }}

  {% elif message.role == 'assistant' and message.content is not none %}
    ASSISTANT: {{ message.content.strip() }}
    {{ '\n' }}
  {% endif %}

{% endfor %}

{% if add_generation_prompt %}
  ASSISTANT: 
{% endif %}'''

model_path = "mlx-community/llava-1.5-7b-4bit"
tokenizer_config = {"chat_template": llava_1_5_chat_template}
model, processor = load(model_path, tokenizer_config)

However, when running generate, I now get the following error, which I suppose will be fixed in https://github.com/Blaizzy/mlx-vlm/pull/67.

TypeError: Model.__call__() got an unexpected keyword argument 'image_grid_thw'

[Blaizzy]

Hey @DePasqualeOrg

I managed to replicate the issue and will fix it in #67

Yesterday, I accidentally introduced a bug on v0.0.14 and v0.0.15.

[Blaizzy]

This template issue is being cause by transformers :(

[Blaizzy]

I have to reupload all models again.

They moved the templates to the preprocessor config file.