Closed Kaiseem closed 1 month ago
Hi @Kaiseem, we've moved active development on these pathways to vllm-project/llm-compressor. The example you are trying has an out of date recipe - you could try the updated examples here: https://github.com/vllm-project/llm-compressor/tree/main/examples
Hello @Kaiseem As there have been no further comments on this thread, I am going to go ahead and close out this issue. Feel free to re-open it as needed. We hope you're able to try the updated examples provided by @bfineran! Best, Jeannie / Neural Magic
Hi, i try your example from the main page:
git clone https://github.com/neuralmagic/sparseml pip install -e "sparseml[transformers]" wget https://huggingface.co/neuralmagic/TinyLlama-1.1B-Chat-v0.4-pruned50-quant-ds/raw/main/recipe.yaml sparseml.transformers.text_generation.oneshot --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --dataset open_platypus --recipe recipe.yaml --output_dir ./obcq_deployment --precision float16
2024-07-10 20:18:39 sparseml.transformers.finetune.runner INFO One Shot 2024-07-10 20:18:43 sparseml.core.recipe.recipe INFO Loading recipe from file recipe.yaml Traceback (most recent call last): File "/opt/conda/bin/sparseml.transformers.text_generation.oneshot", line 8, in
sys.exit(oneshot())
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/text_generation.py", line 77, in oneshot
main(model_args, data_args, training_args)
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/text_generation.py", line 367, in main
stage_runner.one_shot()
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/runner.py", line 174, in one_shot
self.trainer.one_shot(calib_data, stage=stage)
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/transformers/finetune/session_mixin.py", line 409, in one_shot
sparseml.apply(
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/session.py", line 526, in apply
return active_session().apply(
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/session.py", line 242, in apply
self.initialize(kwargs)
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/session.py", line 187, in initialize
mod_data = self._lifecycle.initialize(
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/lifecycle/session.py", line 94, in initialize
self._check_compile_recipe()
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/lifecycle/session.py", line 175, in _check_compile_recipe
self.modifiers = self.recipe_container.compiled_recipe.create_modifier(
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/recipe/recipe.py", line 387, in create_modifier
stage_modifiers = stage.create_modifier(framework)
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/recipe/stage.py", line 137, in create_modifier
modifier = modifier.create_modifier(framework)
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/recipe/modifier.py", line 94, in create_modifier
return ModifierFactory.create(
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/factory.py", line 150, in create
return ModifierFactory._mainregistry[type](framework=framework, kwargs)
File "/ossfs/workspace/Sparse_ML_proj/sparseml/src/sparseml/core/framework_object.py", line 63, in new
return MultiFrameworkObject.load_framework_class(
File "/opt/conda/lib/python3.8/site-packages/pydantic/main.py", line 176, in init
self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for QuantizationModifierPyTorch
config_groups
Field required [type=missing, input_value={'ignore': ['LlamaRotaryE..., 'symmetric': False}}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.7/v/missing
it means that config_groups should be in yaml file, but i cannot find config_groups in the yaml file. Do you have any suggestion? Best wishes.