search

DFKI-NLP / perseus-textgen

A repository for scripts to run awesomely large language models with text generation inference APIs and (chat) UIs
MIT License
0 stars 0 forks source link

consider vLLM #4

Open ArneBinder opened 1 year ago

ArneBinder commented 1 year ago

motivation: https://github.com/vllm-project/vllm#performance