微调gemma，报错KeyError: 'target_mask'

[CyrusLiu158]

无论是使用官方的jsonl，还是我自己的。都是报错： 0%| | 0/27 [00:00<?, ?it/s]Traceback (most recent call last): File "/home/seatune/桌面/workspace/Firefly/train.py", line 400, in main() File "/home/seatune/桌面/workspace/Firefly/train.py", line 388, in main train_result = trainer.train() File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer.py", line 1624, in train return inner_training_loop( File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer.py", line 1928, in _inner_training_loop for step, inputs in enumerate(epoch_iterator): File "/home/seatune/anaconda3/lib/python3.10/site-packages/accelerate/data_loader.py", line 452, in iter current_batch = next(dataloader_iter) File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 634, in next data = self._next_data() File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 678, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch return self.collate_fn(data) File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer_utils.py", line 773, in call return self.data_collator(features) File "/home/seatune/桌面/workspace/Firefly/component/collator.py", line 24, in call target_mask = x['target_mask'] KeyError: 'target_mask' 0%| | 0/27 [00:00<?, ?it/s] Training completed.

我的配置文件： { "output_dir": "./training_outputs", "model_name_or_path": "google/gemma-1.1-7b-it", "train_file": "./jsonl/data_inspiration_0408.jsonl", "template_name":"gemma", "num_train_epochs": 3, "per_device_train_batch_size": 3, "save_steps": 10000, "logging_dir": "./logs", "learning_rate": 5e-6, "max_seq_length": 1024, "logging_steps": 100, "save_total_limit": 2, "seed": 42, "lora_rank":32, "lora_alpha":16, "lora_dropout":0.1, "fp16": true, "bf16": false }

[kunling-cxk]

waiting，i have the same question

[kunling-cxk]

无论是使用官方的jsonl，还是我自己的。都是报错： 0%| | 0/27 [00:00<?, ?it/s]Traceback (most recent call last): File "/home/seatune/桌面/workspace/Firefly/train.py", line 400, in main() File "/home/seatune/桌面/workspace/Firefly/train.py", line 388, in main train_result = trainer.train() File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer.py", line 1624, in train return inner_training_loop( File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer.py", line 1928, in _inner_training_loop for step, inputs in enumerate(epoch_iterator): File "/home/seatune/anaconda3/lib/python3.10/site-packages/accelerate/data_loader.py", line 452, in iter current_batch = next(dataloader_iter) File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 634, in next data = self._next_data() File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 678, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch return self.collate_fn(data) File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer_utils.py", line 773, in call return self.data_collator(features) File "/home/seatune/桌面/workspace/Firefly/component/collator.py", line 24, in call target_mask = x['target_mask'] KeyError: 'target_mask' 0%| | 0/27 [00:00<?, ?it/s] Training completed.

我的配置文件： { "output_dir": "./training_outputs", "model_name_or_path": "google/gemma-1.1-7b-it", "train_file": "./jsonl/data_inspiration_0408.jsonl", "template_name":"gemma", "num_train_epochs": 3, "per_device_train_batch_size": 3, "save_steps": 10000, "logging_dir": "./logs", "learning_rate": 5e-6, "max_seq_length": 1024, "logging_steps": 100, "save_total_limit": 2, "seed": 42, "lora_rank":32, "lora_alpha":16, "lora_dropout":0.1, "fp16": true, "bf16": false }

hi,do you have solved the problem?