search

predibase / lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
https://loraexchange.ai
Apache License 2.0
2.13k stars 139 forks source link

Add Support for AutoModelForSequenceClassification Models #509

Open akkky02 opened 3 months ago

akkky02 commented 3 months ago

Feature request

LoRAX currently only supports text generation models (e.g., causal language models). It lacks support for sequence classification models like AutoModelForSequenceClassification. This issue proposes adding support for AutoModelForSequenceClassification models to LoRAX. This would enable users to serve and perform inference with fine-tuned sequence classification models. This would expand LoRAX's capabilities, allowing users to serve a wider range of models and perform tasks like sentiment analysis, topic classification, and more.

Motivation

Currently, users who want to serve sequence classification models with LoRAX have to rely on external inference APIs or implement their own solutions. This can be cumbersome and less efficient. Adding native support for AutoModelForSequenceClassification would streamline the process and make LoRAX more versatile.

Your contribution

I'm willing to contribute to this feature.

jgolebiowski commented 1 month ago

+1 This would be very useful.