Open himanshushukla12 opened 1 month ago
Are you trying to use qlora? Can you try python llama2.py --model_name meta-llama/Llama-2-7b-hf --qlora
?
python llama2.py --model_name meta-llama/Llama-2-7b-hf --qlora
Here is the complete output
/Olive/examples/llama2$ python llama2.py --model_name meta-llama/Llama-2-7b-hf --qlora
[2024-08-21 10:14:32,863] [INFO] [config.py:252:validate_evaluate_input_model] No evaluator is specified, skip to evaluate model
[2024-08-21 10:14:32,864] [INFO] [run.py:138:run_engine] Running workflow default_workflow
[2024-08-21 10:14:32,864] [INFO] [cache.py:51:__init__] Using cache directory: /home/z004x2xz/WorkAssignedByMatt/Olive/examples/llama2/.olive-cache/default_workflow
[2024-08-21 10:14:32,865] [INFO] [engine.py:1012:save_olive_config] Saved Olive config to /home/z004x2xz/WorkAssignedByMatt/Olive/examples/llama2/.olive-cache/default_workflow/olive_config.json
[2024-08-21 10:14:33,733] [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-cuda
[2024-08-21 10:14:33,741] [INFO] [engine.py:275:run] Running Olive on accelerator: gpu-cuda
[2024-08-21 10:14:33,741] [INFO] [engine.py:1109:_create_system] Creating target system ...
[2024-08-21 10:14:33,741] [INFO] [engine.py:1112:_create_system] Target system created in 0.000035 seconds
[2024-08-21 10:14:33,741] [INFO] [engine.py:1121:_create_system] Creating host system ...
[2024-08-21 10:14:33,741] [INFO] [engine.py:1124:_create_system] Host system created in 0.000026 seconds
[2024-08-21 10:14:33,767] [INFO] [engine.py:877:_run_pass] Running pass f:QLoRA
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:05<00:00, 2.55s/it]
You are calling `save_pretrained` to a 4-bit converted model, but your `bitsandbytes` version doesn't support it. If you want to save 4-bit models, make sure to have `bitsandbytes>=0.41.3` installed.
Generating train split: 0 examples [00:01, ? examples/s]
[2024-08-21 10:14:43,729] [ERROR] [engine.py:974:_run_pass] Pass run failed.
Traceback (most recent call last):
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1750, in _prepare_split_single
for key, record in generator:
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/packaged_modules/generator/generator.py", line 32, in _generate_examples
for idx, ex in enumerate(self.config.generator(**gen_kwargs)):
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 257, in data_generator
dataset = data_container.pre_process(data_container.load_dataset())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/data/container/data_container.py", line 26, in load_dataset
return self.config.load_dataset(**self.config.load_dataset_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/data/component/load_dataset.py", line 39, in huggingface_dataset
dataset = load_dataset(path=data_name, name=subset, data_dir=data_dir, split=split, data_files=data_files, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 2594, in load_dataset
builder_instance = load_dataset_builder(
^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 2266, in load_dataset_builder
dataset_module = dataset_module_factory(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 1914, in dataset_module_factory
raise e1 from None
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 1896, in dataset_module_factory
).get_module()
^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 1225, in get_module
standalone_yaml_path = cached_path(
^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/utils/file_utils.py", line 201, in cached_path
output_path = get_from_cache(
^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/utils/file_utils.py", line 635, in get_from_cache
raise ConnectionError(f"Couldn't reach {url} (error {response.status_code})")
ConnectionError: Couldn't reach https://huggingface.co/datasets/nampdn-ai/tiny-codes/resolve/9aebe5ee8b406356d5f5f2d603bc0a1684ee8ce7/.huggingface.yaml (error 403)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/engine/engine.py", line 962, in _run_pass
output_model_config = host.run_pass(p, input_model_config, output_model_path, pass_search_point)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/systems/local.py", line 30, in run_pass
output_model = the_pass.run(model, output_model_path, point)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/olive_pass.py", line 206, in run
output_model = self._run_for_config(model, config, output_model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 688, in _run_for_config
output_model = self.train_and_save_new_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 483, in train_and_save_new_model
train_dataset, eval_dataset = self.get_datasets(config)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 269, in get_datasets
train_dataset = Dataset.from_generator(data_generator, gen_kwargs={"data_config": train_data_config})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 1130, in from_generator
).read()
^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/io/generator.py", line 47, in read
self.builder.download_and_prepare(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1029, in download_and_prepare
self._download_and_prepare(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1791, in _download_and_prepare
super()._download_and_prepare(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1124, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1629, in _prepare_split
for job_id, done, content in self._prepare_split_single(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1786, in _prepare_split_single
raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.exceptions.DatasetGenerationError: An error occurred while generating the dataset
[2024-08-21 10:14:43,763] [WARNING] [engine.py:370:run_accelerator] Failed to run Olive on gpu-cuda.
Traceback (most recent call last):
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1750, in _prepare_split_single
for key, record in generator:
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/packaged_modules/generator/generator.py", line 32, in _generate_examples
for idx, ex in enumerate(self.config.generator(**gen_kwargs)):
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 257, in data_generator
dataset = data_container.pre_process(data_container.load_dataset())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/data/container/data_container.py", line 26, in load_dataset
return self.config.load_dataset(**self.config.load_dataset_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/data/component/load_dataset.py", line 39, in huggingface_dataset
dataset = load_dataset(path=data_name, name=subset, data_dir=data_dir, split=split, data_files=data_files, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 2594, in load_dataset
builder_instance = load_dataset_builder(
^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 2266, in load_dataset_builder
dataset_module = dataset_module_factory(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 1914, in dataset_module_factory
raise e1 from None
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 1896, in dataset_module_factory
).get_module()
^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/load.py", line 1225, in get_module
standalone_yaml_path = cached_path(
^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/utils/file_utils.py", line 201, in cached_path
output_path = get_from_cache(
^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/utils/file_utils.py", line 635, in get_from_cache
raise ConnectionError(f"Couldn't reach {url} (error {response.status_code})")
ConnectionError: Couldn't reach https://huggingface.co/datasets/nampdn-ai/tiny-codes/resolve/9aebe5ee8b406356d5f5f2d603bc0a1684ee8ce7/.huggingface.yaml (error 403)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/engine/engine.py", line 349, in run_accelerator
output_footprint = self.run_no_search(
^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/engine/engine.py", line 441, in run_no_search
should_prune, signal, model_ids = self._run_passes(
^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/engine/engine.py", line 814, in _run_passes
model_config, model_id, output_model_hash = self._run_pass(
^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/engine/engine.py", line 962, in _run_pass
output_model_config = host.run_pass(p, input_model_config, output_model_path, pass_search_point)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/systems/local.py", line 30, in run_pass
output_model = the_pass.run(model, output_model_path, point)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/olive_pass.py", line 206, in run
output_model = self._run_for_config(model, config, output_model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 688, in _run_for_config
output_model = self.train_and_save_new_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 483, in train_and_save_new_model
train_dataset, eval_dataset = self.get_datasets(config)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/olive/passes/pytorch/lora.py", line 269, in get_datasets
train_dataset = Dataset.from_generator(data_generator, gen_kwargs={"data_config": train_data_config})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 1130, in from_generator
).read()
^^^^^^
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/io/generator.py", line 47, in read
self.builder.download_and_prepare(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1029, in download_and_prepare
self._download_and_prepare(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1791, in _download_and_prepare
super()._download_and_prepare(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1124, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1629, in _prepare_split
for job_id, done, content in self._prepare_split_single(
File "/home/z004x2xz/WorkAssignedByMatt/Olive/venv3.11/lib/python3.11/site-packages/datasets/builder.py", line 1786, in _prepare_split_single
raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.exceptions.DatasetGenerationError: An error occurred while generating the dataset
[2024-08-21 10:14:43,768] [INFO] [engine.py:292:run] Run history for gpu-cuda:
[2024-08-21 10:14:43,771] [INFO] [engine.py:587:dump_run_history] run history:
+------------+-------------------+-------------+----------------+-----------+
| model_id | parent_model_id | from_pass | duration_sec | metrics |
+============+===================+=============+================+===========+
| 1ea91e35 | | | | |
+------------+-------------------+-------------+----------------+-----------+
[2024-08-21 10:14:43,772] [INFO] [engine.py:307:run] No packaging config provided, skip packaging artifacts
Can you login huggingface first? https://huggingface.co/docs/huggingface_hub/en/guides/cli#huggingface-cli-login
You need to gain access to tiny code data set. https://huggingface.co/datasets/nampdn-ai/tiny-codes
While executing the file in folder
Olive/examples/llama2
i got the error of TypeError: LlamaForCausalLM.forward() got an unexpected keyword argument 'past_key_values.0.key' while executing :python llama2.py --model_name meta-llama/Llama-2-7b-hf
Olive config Add Olive configurations here.
Olive logs Add logs here.
Other information
Additional context I build olive from source using
pip install -e .
Please help