search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
29.35k stars 4.39k forks source link

[Misc]: how to add tests for new backends? #7683

Open ilya-lavrenov opened 2 months ago

ilya-lavrenov commented 2 months ago

Anything you want to discuss about vllm.

I see that some backends like Intel CPU have tests, but I cannot find any such configurations in .buildkite folder. Could you please elaborate how we can add some simple scripts to check that OpenVINO backend is not broken?

We have simple script for it https://github.com/vllm-project/vllm/blob/main/.buildkite/run-openvino-test.sh, but don't know how to add it to CI service.

Thanks

DarkLight1337 commented 2 months ago

@WoosukKwon perhaps you have a more detailed idea about this?