Activation Ordering Tests - Githubissues

vllm-project / llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Apache License 2.0

644 stars 52 forks source link

Activation Ordering Tests #135

Closed kylesayrs closed 2 months ago

kylesayrs commented 2 months ago

Purpose

Test new activation ordering options
Upload activation ordered models to hugging face for testing on vllm
Support e2e tests with gsm8k calibration dataset

Changes

Modified parse_params to recursively search directories for config yamls
Added preprocess_tokenize_dataset helper function to support different dataset preprocessing
Added nightly tests to test compression of models with all activation ordering strategies (None, weight, group)
Fixed bug in _validate_test_config

Testing

Added tests require ct changes to land in main, but pass with a local installation of the ct changes
Made sure that the modified parse_params function is compatible with existing use cases