search

armbues / SiLLM

SiLLM simplifies the process of training and running Large Language Models (LLMs) on Apple Silicon by leveraging the MLX framework.
MIT License
209 stars 21 forks source link

AttributeError: 'str' object has no attribute 'apply_chat_template' #4

Closed alew3 closed 5 months ago

alew3 commented 5 months ago

I'm trying to get llama-3 working , but when I type anything .. it loads the model and gives this error:

AttributeError: 'str' object has no attribute 'apply_chat_template'

2024-04-19 14:41:24 - 'str' object has no attribute 'apply_chat_template' Traceback (most recent call last): File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/chainlit/utils.py", line 39, in wrapper return await user_function(**params_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/username/projects/ask_azion/SiLLM/app/app.py", line 133, in on_message prompt = conversation.add_user(message.content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/sillm/core/conversation.py", line 81, in add_user """ File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/sillm/core/conversation.py", line 67, in add_message File "/Users/username/anaconda3/envs/mlx/lib/python3.11/site-packages/sillm/core/conversation.py", line 23, in apply_chat_template return self.template.apply_chat_template(messages=self.messages, add_generation_prompt=add_generation_prompt) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'apply_chat_template'

I'm using anaconda for the python environment.

armbues commented 5 months ago

Are you running the latest version 0.1.2 with an updated app.py from the repository? This is an error that was fixed since Llama-3 came out.

Try upgrading with pip install sillm-mlx --upgrade and also update the repo using git pull origin.

I just checked with the latest source & build and Llama-3 8B definitely works.

alew3 commented 5 months ago

I tried upgrading , and starting from scratch with a new environment .. still getting the error. Not sure what it could be.

my steps to reproduce: $ conda create -n "sillm" python=3.11 $ conda activate sillm $ git clone https://github.com/armbues/SiLLM.git $ cd SiLLM/app $ pip install sillm-mlx $ pip install -r requirements.txt created .env file and setting the SILLM_MODEL_DIR environment variable within the /app folder $ python -m chainlit run app.py -w

selected the model "llama-3-8b-4bit" and chat template "llama3"

image

and I still get the error: AttributeError: 'str' object has no attribute 'apply_chat_template'

image

armbues commented 5 months ago

Looks like I had not pushed the fixed code yet. Sorry for the oversight!

Can you pull the latest app.py and try again?

alew3 commented 5 months ago

Yep, working now! Thanks!