Closed silvacarl2 closed 7 months ago
What would you suggest exactly? Could you please detail this?
without having to create a fine tuned classification model, "facebook/bart-large-mnli" does a very good job. as part of a vllm "toolkit", if it supports mistralai/Mistral-7B-Instruct-v0.1, whisper, and facebook/bart-large-mnli, all running blazing fast, ANYTHING can be built out of it. Literally anything.
if through testing mistralai/Mistral-7B-Instruct-v0.1 can do as well or better than facebook/bart-large-mnli at categorization, then it would be uneccsary but we have not proved yet if that is the case or not.
right now mistralai/Mistral-7B-Instruct-v0.1 appears to be the best 7B model we have seen, so we are going to test it witlh vllm.
another huge problem requiring speed is BartForSequenceClassification ala "facebook/bart-large-mnli"