search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
28.98k stars 4.32k forks source link

[New Model]: support for BartForSequenceClassification #5985

Open Sapessii opened 3 months ago

Sapessii commented 3 months ago

The model to consider.

Hi!

Is there any plan on supporting https://huggingface.co/facebook/bart-large-mnli on vllm? When I try to run it it says "BartForSequenceClassification" is not supported now, do you know any alternatives or if I'm doing something wrong?

Thank you

The closest model vllm already supports.

No response

What's your difficulty of supporting the model you want?

No response

ashleykleynhans commented 3 months ago

Would be great if support could be added for BartForSequenceClassification.