One-Shot LLM Compression bug with obcq recipe.yaml

[Kaiseem]

Hi, i try your example from the main page:

git clone https://github.com/neuralmagic/sparseml pip install -e "sparseml[transformers]" wget https://huggingface.co/neuralmagic/TinyLlama-1.1B-Chat-v0.4-pruned50-quant-ds/raw/main/recipe.yaml sparseml.transformers.text_generation.oneshot --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --dataset open_platypus --recipe recipe.yaml --output_dir ./obcq_deployment --precision float16

2024-07-10 20:18:39 sparseml.transformers.finetune.runner INFO One Shot 2024-07-10 20:18:43 sparseml.core.recipe.recipe INFO Loading recipe from file recipe.yaml Traceback (most recent call last): File "/opt/conda/bin/sparseml.transformers.text_generation.oneshot", line 8, in sys.exit(oneshot()) File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/text_generation.py", line 77, in oneshot main(model_args, data_args, training_args) File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/text_generation.py", line 367, in main stage_runner.one_shot() File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/runner.py", line 174, in one_shot self.trainer.one_shot(calib_data, stage=stage) File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/session_mixin.py", line 409, in one_shot sparseml.apply( File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/session.py", line 526, in apply return active_session().apply( File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/session.py", line 242, in apply self.initialize(kwargs) File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/session.py", line 187, in initialize mod_data = self._lifecycle.initialize( File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/lifecycle/session.py", line 94, in initialize self._check_compile_recipe() File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/lifecycle/session.py", line 175, in _check_compile_recipe self.modifiers = self.recipe_container.compiled_recipe.create_modifier( File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/recipe/recipe.py", line 387, in create_modifier stage_modifiers = stage.create_modifier(framework) File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/recipe/stage.py", line 137, in create_modifier modifier = modifier.create_modifier(framework) File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/recipe/modifier.py", line 94, in create_modifier return ModifierFactory.create( File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/factory.py", line 150, in create return ModifierFactory._mainregistry[type](framework=framework, kwargs) File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/framework_object.py", line 63, in new return MultiFrameworkObject.load_framework_class( File "/opt/conda/lib/python3.8/site-packages/pydantic/main.py", line 176, in init self.__pydantic_validator__.validate_python(data, self_instance=self) pydantic_core._pydantic_core.ValidationError: 1 validation error for QuantizationModifierPyTorch config_groups Field required [type=missing, input_value={'ignore': ['LlamaRotaryE..., 'symmetric': False}}}}, input_type=dict] For further information visit https://errors.pydantic.dev/2.7/v/missing

it means that config_groups should be in yaml file, but i cannot find config_groups in the yaml file. Do you have any suggestion? Best wishes.

[bfineran]

Hi @Kaiseem, we've moved active development on these pathways to vllm-project/llm-compressor. The example you are trying has an out of date recipe - you could try the updated examples here: https://github.com/vllm-project/llm-compressor/tree/main/examples

[jeanniefinks]

Hello @Kaiseem As there have been no further comments on this thread, I am going to go ahead and close out this issue. Feel free to re-open it as needed. We hope you're able to try the updated examples provided by @bfineran! Best, Jeannie / Neural Magic