Please check that this issue hasn't been reported before.
[X] I searched previous Bug Reports didn't find any similar reports.
Expected Behavior
It should run without an error, as it does when you have micro_batch_size and eval_batch_size set to 1.
Current behaviour
Returns two errors;
ValueError: expected sequence of length 406 at dim 1 (got 75)
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (rejected_input_ids in this case) have excessive nesting (inputs type list where type int is expected).
Traceback (most recent call last):
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 759, in convert_to_tensors
tensor = as_tensor(value)
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 721, in as_tensor
return torch.tensor(value)
ValueError: expected sequence of length 406 at dim 1 (got 75)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/media/xzuyn/NVMe/LClones/axolotl/src/axolotl/cli/train.py", line 59, in <module>
fire.Fire(do_cli)
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/fire/core.py", line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/fire/core.py", line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/media/xzuyn/NVMe/LClones/axolotl/src/axolotl/cli/train.py", line 35, in do_cli
return do_train(parsed_cfg, parsed_cli_args)
File "/media/xzuyn/NVMe/LClones/axolotl/src/axolotl/cli/train.py", line 55, in do_train
return train(cfg=cfg, cli_args=cli_args, dataset_meta=dataset_meta)
File "/media/xzuyn/NVMe/LClones/axolotl/src/axolotl/train.py", line 160, in train
trainer.train(resume_from_checkpoint=resume_from_checkpoint)
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/transformers/trainer.py", line 1780, in train
return inner_training_loop(
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/transformers/trainer.py", line 2085, in _inner_training_loop
for step, inputs in enumerate(epoch_iterator):
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/accelerate/data_loader.py", line 451, in __iter__
current_batch = next(dataloader_iter)
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 631, in __next__
data = self._next_data()
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 675, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/media/xzuyn/NVMe/LClones/axolotl/src/axolotl/monkeypatch/data/batch_dataset_fetcher.py", line 32, in fetch
return self.collate_fn(data)
File "/media/xzuyn/NVMe/LClones/axolotl/src/axolotl/utils/collators.py", line 106, in __call__
features = self.tokenizer.pad(
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 3369, in pad
return BatchEncoding(batch_outputs, tensor_type=return_tensors)
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 224, in __init__
self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
File "/media/xzuyn/NVMe/LClones/axolotl/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 775, in convert_to_tensors
raise ValueError(
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`rejected_input_ids` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
Steps to reproduce
Run the YAML provided, which has a micro_batch_size and eval_batch_size of 2.
Please check that this issue hasn't been reported before.
Expected Behavior
It should run without an error, as it does when you have
micro_batch_size
andeval_batch_size
set to 1.Current behaviour
Returns two errors;
Steps to reproduce
Run the YAML provided, which has a
micro_batch_size
andeval_batch_size
of 2.I tested:
Config yaml
Possible solution
No response
Which Operating Systems are you using?
Python Version
3.10.12
axolotl branch-commit
main/bda48f0
Acknowledgements