Run python main_dpo.py --template "vicuna_v1.1"

[Addss-s]

File "/home/lm/OpenFedLLM-main/main_dpo.py", line 109, in results = trainer.train() File "/home/lm/yes/envs/opfl/lib/python3.10/site-packages/transformers/trainer.py", line 1539, in train return inner_training_loop( File "/home/lm/yes/envs/opfl/lib/python3.10/site-packages/transformers/trainer.py", line 1787, in _inner_training_loop for step, inputs in enumerate(epoch_iterator): File "/home/lm/yes/envs/opfl/lib/python3.10/site-packages/accelerate/data_loader.py", line 384, in iter current_batch = next(dataloader_iter) File "/home/lm/yes/envs/opfl/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 633, in next data = self._next_data() File "/home/lm/yes/envs/opfl/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 677, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/lm/yes/envs/opfl/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch return self.collate_fn(data) File "/home/lm/yes/envs/opfl/lib/python3.10/site-packages/trl/trainer/utils.py", line 471, in call prompt = feature["prompt"] KeyError: 'prompt' 0%| | 0/10 [00:00<?, ?it/s] Why is this error reported

[rui-ye]

what dataset are you using?

[Addss-s]

Using the default one in the code and CUDA_VISIBLE_DEVICES=1 python main_sft.py \ --model_name_or_path "meta-llama/Llama-2-7b-hf" \ --dataset_name "vicgalle/alpaca-gpt4" \ --dataset_sample 20000 \ --fed_alg "fedavg" \ --num_clients 20 \ --sample_clients 2 \ --max_steps 10 \ --num_rounds 200 \ --batch_size 16 \ --gradient_accumulation_steps 1 \ --seq_length 512 \ --peft_lora_r 32 \ --peft_lora_alpha 64 \ --use_peft \ --load_in_8bit \ --output_dir "./output" \ --template "alpaca" \This can run correctly

[rui-ye]

Please carefully check you script. Are you running dpo or sft?

[rui-ye]

In you initial question, you are running main_dpo.py. But your latest response shows that you are running main_sft.py

[Addss-s]

What I mean is that main_stft. py can run normally when running, but when running main_depo. py, it displays the error above

[rui-ye]

so what dataset are you using for running main_dpo.py? Still "vicgalle/alpaca-gpt4"?

[Addss-s]

yes

[rui-ye]

Running DPO requires that each data sample consists (query, preferred response, dis-referred response). So, you cannot use "vicgalle/alpaca-gpt4" dataset. Please refer to the usage cases of ultrafeedback and hh-rlhf in our code and paper.

[Addss-s]

Now the default dataset is used, which is called lucasmccabe-lmi___parquet/lucasmccabe-lmi--CodeAlpaca-20k-b0821000b926cd76/0.0.0/14a00e99c0d15a23649d0db8944380ac81082d4b021f398733dd84f3a6c569a7/cache-9802f004d06beca7.arrow

[rui-ye]

there is something wrong with your default setup. Please take a look at this: https://github.com/rui-ye/OpenFedLLM/blob/main/training_scripts/run_dpo.sh

[Addss-s]

Thank you for your answer! I have successfully run it.