[Bug] Issue with `pack_image_features()` in LlavaNextForConditionalGeneration: Missing `vision_feature_select_strategy` Argument

Checklist

[X] 1. I have searched related issues but cannot get the expected help.
[X] 2. The bug has not been fixed in the latest version.
[ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

I am using Docker to serve a fine-tuned LLaVA-Next model with the following script:

docker run --runtime nvidia --gpus '"device=2"' \
        -v /srv/model-repository/LLMs/meta:/root/.cache/huggingface \
        -p 23333:23333 \
        --ipc=host \
        openmmlab/lmdeploy:v0.6.3.post1-cu11 \
        lmdeploy serve api_server ben81828/PULSE-7B-HF \
            --model-name llava-v1 \
            --chat-template llava-v1

It works fine when I send a text-only message. However, when I call it with both text and image, I get the following error:

TypeError: LlavaNextForConditionalGeneration.pack_image_features() missing 1 required positional argument: 'vision_feature_select_strategy'

After inspecting the code in lmdeploy/lmdeploy/vl/model/llava_next.py, I found the following line:

image_features, feature_lens = self.model.pack_image_features(
    image_features, 
    image_sizes,
    image_newline=self.model.image_newline,
)

I modified it by adding the vision_feature_select_strategy argument as follows:

image_features, feature_lens = self.model.pack_image_features(
    image_features, 
    image_sizes,
    vision_feature_select_strategy=self.hf_config.vision_feature_select_strategy,
    image_newline=self.model.image_newline,
)

This change resolved the issue, and it works fine now.

I am curious if there is a better way to fix this or if I missed something. I did not see any similar issue reported in the repository, so I wanted to bring it up here.

Thank you!

Reproduction

This is the code I use to call the LLaVA service.

from openai import OpenAI
client = OpenAI(
    base_url="http://10.18.27.125:23333/v1",
    api_key="aicenter",
)

import base64

# Function to encode the image
def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode('utf-8')

# Path to your image
image_path = "ecg_1.png"

# Getting the base64 string
base64_image = encode_image(image_path)

completion = client.chat.completions.create(
    model="llava-v1", # lmdeploy的model-name
    messages=[
        {"role": "user", 
         "content": [
             { 
                 "type": "text",
                 "text": f"tell me details about this image."
             },
             {
                 "type": "image_url",
                 "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}
             },
         ],
        },
  ]
)

Environment

--

Error traceback

**

InternLM / lmdeploy