search

vllm-project / llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
Apache License 2.0
644 stars 52 forks source link

Validate Recipe Parsing Output #100

Closed kylesayrs closed 1 month ago

kylesayrs commented 2 months ago

Previously there was an issue with tests which resulted in test yaml paths being read incorrectly. The current behavior is that the path string is parsed as if it was a yaml. This returns a str type which is not only wrong content but leads to type conflicts downstream.

Now the user gets a clean error

Could not parse recipe from string tests/llmcompressor/transformers/obcq/recipes/quant_and_sparse.yaml. If you meant load from a file, please make sure that the specified file path exists