Support for Phi-3 small and Phi-3 medium? Multimodal support?

[coder543]

Since Phi-3 mini did so well on the leaderboard, it would be interesting to see where the new small and medium models land.

With Phi-3 vision, it also seems like we're starting to have a pretty healthy selection of multimodal models available, and it would be cool to see the Chatbot Arena support that at some point too.

[brandonbiggs]

Not an answer to your issue, but trying to help. I've been trying to figure out the prompt template for these in conversation.py, but I'm missing something still as I'm just getting junk output:

register_conv_template(
    Conversation(
        name="phi3",
        roles=("<|user|>", "<|assistant|>"),
        sep_style=SeparatorStyle.DEFAULT,
        sep="<|end|>",
    )
)

The example from the documentation I'm trying to match:

<|user|>
I am going to Paris, what should I see?<|end|>
<|assistant|>
Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:\n\n1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.\n2. The Louvre Museum: The Louvre is one of the world's largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.\n3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.\n\nThese are just a few of the many attractions that Paris has to offer. With so much to see and do, it's no wonder that Paris is one of the most popular tourist destinations in the world."<|end|>
<|user|>
What is so great about #1?<|end|>
<|assistant|>

[ccdv-ai]

@brandonbiggs did you figure out the right template?

[brandonbiggs]

@ccdv-ai unfortunately no. I gave up and switched to other things. If you figure it out though, please share as I would still like to use Phi3.

[ShiJiawenwen]

I try to solve the problem using the following code, I change the CHATML: elif self.sep_style == SeparatorStyle.CHATML: ret = "<|endoftext|>" if self.system_message: ret += system_prompt + self.sep + "\n" else: ret += "" for role, message in self.messages: if message: if type(message) is tuple: message, images = message message = IMAGE_PLACEHOLDER_STR * len(images) + message ret += role + "\n" + message + self.sep + "\n" else: ret += role + "\n" return ret

register_conv_template( Conversation( name="phi-3", system_template="<|system|>\n{system_message}", roles=("<|user|>", "<|assistant|>"), sep_style=SeparatorStyle.CHATML, sep="<|end|>", stop_token_ids=[100257,100266], stop_str="<|endoftext|>", ) )