[Usage]: Model architectures ['LlavaNextForConditionalGeneration'] are not supported for now

[BITDDD]

Your current environment

vllm==0.6.1.post1 vllm-flash-attn==2.6.1

How would you like to use vllm

I want use LLAVA-1.6-hf-mistral-7B，Tips Model architectures ['LlavaNextForConditionalGeneration'] are not supported for now，but LlavaNextForConditionalGeneration in the Supported architectures.

Before submitting a new issue...

• [X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

Are there any exception stack traces being printed before this error? Maybe vLLM failed to import the model.

[BITDDD]

Are there any exception stack traces being printed before this error? Maybe vLLM failed to import the model.

[DarkLight1337]

Hmm... can you try updating your vLLM version and see if the issue persists?

[BITDDD]

Hmm... can you try updating your vLLM version and see if the issue persists? I Try pip install -U vllm，now version vllm--0.6.3.post1，the same question occur ...

[DarkLight1337]

Are there any exception stack traces being printed before this error? Maybe vLLM failed to import the model.

Can you show the full logs in this cell? Not just the error.

[BITDDD]

Are there any exception stack traces being printed before this error? Maybe vLLM failed to import the model.

Can you show the full logs in this cell? Not just the error.

`from io import BytesIO

import requests from PIL import Image

from vllm import LLM, SamplingParams import os

def run_llava_next():

llm = LLM(
    model="llava-hf/llava-v1.6-mistral-7b-hf",
    image_token_id=32000,
    image_input_shape="1,3,336,336",
    image_feature_size=1176,
)

prompt = "[INST] " + "<image>" * 1176 + (
    "\nWhat is shown in this image? [/INST]")
url = "https://h2o-release.s3.amazonaws.com/h2ogpt/bigben.jpg"
image = Image.open(BytesIO(requests.get(url).content))
sampling_params = SamplingParams(temperature=0.8,
                                 top_p=0.95,
                                 max_tokens=100)

outputs = llm.generate(
    {
        "prompt": prompt,
        "multi_modal_data": {
            "image": image
        }
    },
    sampling_params=sampling_params)

generated_text = ""
for o in outputs:
    generated_text += o.outputs[0].text

print(f"LLM output:{generated_text}")

if name == "main": run_llava_next()` My code and full logs

[DarkLight1337]

Do you not get any INFO messages in your output? Maybe you have to configure your Jupyter notebook to output those...

[BITDDD]

Do you not get any INFO messages in your output? Maybe you have to configure your Jupyter notebook to output those...

emmm，It seems that there is no info messages...

[DarkLight1337]

Can you try running this code in a Python script instead of notebook?

[BITDDD]

Can you try running this code in a Python script instead of notebook?

I'll try it out and synchronize new progress here. Thank you for your time.