search

vllm-project / llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
Apache License 2.0
407 stars 29 forks source link

[ DOC ] Simple Example with Mistral #96

Closed robertgshaw2-neuralmagic closed 6 days ago

robertgshaw2-neuralmagic commented 3 weeks ago

SUMMARY:

TEST PLAN: