search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.26k stars 1.58k forks source link

[Bench] Add support for multiple backend #3037

Open cyx-6 opened 5 days ago

cyx-6 commented 5 days ago

This PR adds the support for vllm and llama.cpp backend, especially for json generation.