recipe.yaml not found - Githubissues

botox-100 commented 2 months ago

Hi, i try your example from the main page:

git clone https://github.com/neuralmagic/sparseml pip install -e "sparseml[transformers]" wget https://huggingface.co/neuralmagic/TinyLlama-1.1B-Chat-v0.4-pruned50-quant-ds/raw/main/recipe.yaml sparseml.transformers.text_generation.oneshot --model_name TinyLlama/TinyLlama-1.1B-Chat-v1.0 --dataset_name open_platypus --recipe recipe.yaml --output_dir ./obcq_deployment --precision float16

and i run into several bugs:

the option -model_name is wrong (seems to be -model now)
the dataset seems to be wrong
MOST PROBLEMATIC: The locally downloaded recipe.yaml file is not found by the program (But it is there). I did not find a workaround yet.

I am using actual ubuntu linux and setup a conda environment with python 3.8 (tried also 3.10) for the test.

I am Master student in Germany and wanted to use sparseGPT as a method in my master thesis. Is there a working example currently or another way to get started with evaluation of the method?

Thanks and greetings from Cologne!

jeanniefinks commented 2 months ago

Hi @botox-100 A heads up our ML team is taking a closer look at this. Thank you for bringing this to our attention. More soon, Jeannie / Neural Magic

dbogunowicz commented 2 months ago

Hey @botox-100

Ok, I think the solution is straightforward:

You are right, the displayed CLI commands are incorrect/out of date. Where did you pull those code snippets from? The correct CLI command should be:

sparseml.transformers.text_generation.oneshot --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --dataset open_platypus --recipe recipe.yaml --output_dir ./obcq_deployment --precision float16

I was able to pass recipe.yaml successfuly:

sparseml git:(release/1.7) ✗ sparseml.transformers.text_generation.oneshot --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --dataset open_platypus --recipe recipe.yaml --output_dir ./obcq_deployment --precision float16
2024-04-29 08:17:22 sparseml.transformers.finetune.text_generation WARNING  Process rank: 0, device: cuda:0, n_gpu: 7, distributed training: True, 16-bits training: False
2024-04-29 08:17:22 sparseml.transformers.finetune.text_generation WARNING  Moving TinyLlama/TinyLlama-1.1B-Chat-v1.0 to device cuda:0 for One-Shot
2024-04-29 08:17:27 sparseml.transformers.utils.helpers INFO     model_path is a huggingface model id. Attempting to download recipe from https://huggingface.co/
2024-04-29 08:17:27 sparseml.transformers.utils.helpers INFO     Found recipe: recipe.yaml for model id: TinyLlama/TinyLlama-1.1B-Chat-v1.0. Downloading...
2024-04-29 08:17:27 sparseml.transformers.utils.helpers INFO     Unable to to find recipe recipe.yaml for model id: TinyLlama/TinyLlama-1.1B-Chat-v1.0: 404 Client Error. (Request ID: Root=1-662f5797-5daf3dbd55e97d1c34316a50;1dd2d75e-276d-4969-bb5e-5a611b752f4e)

Entry Not Found for url: https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/resolve/main/recipe.yaml.. Skipping recipe resolution.
2024-04-29 08:17:27 sparseml.transformers.utils.helpers INFO     Failed to infer the recipe from the model_path
Logging all SparseML modifier-level logs to sparse_logs/29-04-2024_08.17.27.log
2024-04-29 08:17:27 sparseml.core.logger.logger INFO     Logging all SparseML modifier-level logs to sparse_logs/29-04-2024_08.17.27.log
Adding labels: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24926/24926 [00:09<00:00, 2768.39 examples/s]
{'train': ['input_ids', 'attention_mask', 'labels']}
/nm/drive0/damian/sparseml/.venv/lib/python3.10/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None)
  warnings.warn(
2024-04-29 08:17:36 sparseml.transformers.finetune.runner INFO     *** One Shot ***
2024-04-29 08:17:37 sparseml.core.recipe.recipe INFO     Loading recipe from file recipe.yaml
2024-04-29 08:17:37 sparseml.modifiers.smoothquant.pytorch INFO     Running LogarithmicEqualizationModifier calibration with 512 samples...
 54%|████████████████████████████████████████████████████████████████████████████████████████████████████                                                                                       | 274/512 [00:06<00:05, 42.26it/s]
...

Are you sure you are properly passing the path to the recipe into the CLI? The argument should properly ingest both absolute and relative path to the file.

Ich hoffe es hilft dir weiter!

dbogunowicz commented 2 months ago

Maybe your problems come from the confusing stdout that I presented above. When you run the same command using the sparseml-nightly, you should get cleaner and less confusing output:

(.venv) ➜  sparseml git:(main) ✗ sparseml.transformers.text_generation.oneshot --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --dataset open_platypus --recipe recipe.yaml --output_dir ./obcq_deployment --precision float16

2024-04-29 08:20:54 sparseml.transformers.finetune.text_generation WARNING  Process rank: 0, device: cuda:0, n_gpu: 7, distributed training: True, 16-bits training: False
2024-04-29 08:20:54 sparseml.transformers.finetune.text_generation WARNING  Moving TinyLlama/TinyLlama-1.1B-Chat-v1.0 to device cuda:0 for One-Shot
2024-04-29 08:20:58 sparseml.transformers.sparsification.modification.modify_model INFO     Modifying the model LlamaForCausalLM to be compatible with SparseML library
Logging all SparseML modifier-level logs to sparse_logs/29-04-2024_08.20.58.log
2024-04-29 08:20:58 sparseml.core.logger.logger INFO     Logging all SparseML modifier-level logs to sparse_logs/29-04-2024_08.20.58.log
Restructuring Platypus Dataset: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24926/24926 [00:01<00:00, 22995.51 examples/s]
Running tokenizer on dataset: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24926/24926 [00:04<00:00, 5488.55 examples/s]
Adding labels: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24926/24926 [00:09<00:00, 2754.65 examples/s]

2024-04-29 08:21:14 sparseml.transformers.finetune.runner INFO     *** One Shot ***
2024-04-29 08:21:14 sparseml.core.recipe.recipe INFO     Loading recipe from file recipe.yaml
2024-04-29 08:21:15 sparseml.modifiers.smoothquant.pytorch INFO     Running LogarithmicEqualizationModifier calibration with 512 samples...
 16%|██████████████████████████████▊                                                                                                                                                             | 84/512 [00:02<00:10, 39.21it/s] 16%|██████████████████████████████▊                                                                                                                                                             | 84/512 [00:02<00:10, 38.92it/s]

jeanniefinks commented 1 week ago

Hi @botox-100 As it's been sometime without a response, we are going to assume you are set for now. Please re-open the thread if you want to continue the conversation, thanks! Jeannie / Neural Magic

neuralmagic / sparseml

recipe.yaml not found #2256

Hi, i try your example from the main page: