[help wanted]: add QwenModel to ci tests

[youkaichao]

Anything you want to discuss about vllm.

see https://github.com/vllm-project/vllm/pull/10262

we should add/replace some model tests for QwenModel

cc @khluu @DarkLight1337

Before submitting a new issue...

• [X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[jeejeelee]

It looks like CI includes QWenVL testing, see:https://github.com/vllm-project/vllm/blob/main/tests/models/decoder_only/vision_language/test_models.py#L336

[DarkLight1337]

As @jeejeelee has mentioned, we do test the model in CI, but to save CI cost, we only test a few models for each PR, while the rest (including Qwen-VL) are only tested nightly.

[DarkLight1337]

That being said, we should add more text-only models to the nightly tests. See #4510

[youkaichao]

by nightly , do you mean the tests in the main branch after merge?

[DarkLight1337]

I'm referring to the special CI build that's only run once per day, e.g.: https://buildkite.com/vllm/ci-aws/builds/11105

[khluu]

@youkaichao To save on cost, we test most of the models in the "Extended" test: https://github.com/vllm-project/vllm/blob/main/.buildkite/test-pipeline.yaml#L355. Every test that is tagged with nightly: true would only show up in the scheduled nightly build: https://buildkite.com/vllm/ci-aws/settings/schedules/7e05f2ee-a1ec-4d42-ad8b-68792efbae2e, including these Extended tests.

[DarkLight1337]

Btw, I've noticed that we can manually unblock nightly tests for Fastcheck CI, but not AWS CI.

Example:

• Fastcheck: https://buildkite.com/vllm/fastcheck/builds/7896
• AWS: https://buildkite.com/vllm/ci-aws/builds/11148

@khluu can you update the pipeline so we can also manually unblock those tests for AWS CI?