EvolvingLMMs-Lab / lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.02k stars 52 forks source link

Update llava.py - Llama 3 Conversation Messages attribute copied as an immutable tuple #76

Open jkitaok opened 1 month ago

jkitaok commented 1 month ago

Fixed Llama 3 Conversation object's messages attribute getting casted as a tuple, from the deepcopy() operation. This allows messages to be appended to messages attribute.

The deepcopy of the conv_templates[self.conv_template] object is as follows:

Conversation(system="<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nA chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.", roles=('<|start_header_id|>user<|end_header_id|>\n\n', '<|start_header_id|>assistant<|end_header_id|>\n\n'), messages=(), offset=0, sep_style=<SeparatorStyle.MPT: 3>, sep='<|eot_id|>', sep2=None, version='llama_3', skip_next=False)

Running an evaluation will cause an error for trying to append messages to a tuple object.

The Conversation objects are instantiated in LLaVA/llava/conversation.py with the messages attribute as a tuple, which conflicts with the append operation in llms_eval/models/llava.py . This PR casts the Conversation.messages attribute as a list for the code to function properly (the messages to be appended to the messages attribute).

Looking forward to hearing back!

kcz358 commented 1 month ago

Hi, please use the newest LLaVA-NeXT to install llava if you want to use the newest llava_llama3 model