BART fine-tuning

[minji-o-j]

best checkpoint기준 실험 (멈춘것 -3) bart finetuning시 batch 또 조정해줘야함

QKV 학습 버전 (tag: QKVo_#num)

• [x] 1번 서버 5 gpu 01

  accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_task1 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_05-16-58_QKVo_1/checkpoint_epoch-11 --train_batch_size=4 --accumulation_steps=24

• cross_task1-_17-03-16
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_task-pc/runs/o7eq711a?workspace=user-minji913

• [x] 5번 서버 5 gpu23

  accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_04-56-03_QKVo_5/checkpoint_epoch-3 --train_batch_size=4 --accumulation_steps=24

• cross_dataset2--59-40

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/bgw7rj4b?workspace=user-minji913
  

• [x] 9번 서버 5

  accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset3_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_04-59-25_QKVo_9/checkpoint_epoch-5 --train_batch_size=4 --accumulation_steps=24

• cross_dataset3_paper--33-22

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/oxxervv3?workspace=user-minji913
  

• [x] 6번 서버 3 gpu 23

  accelerate launch run_textbox.py --model=PTG --dataset=dd  --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-26_04-59-17_QKVo_6/checkpoint_epoch-4 --train_batch_size=2 --accumulation_steps=48

• cross_dataset2--24-06

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/s5pcvxk8?workspace=user-minji913
  

• [x] 10번 서버 3 gpu 23

  accelerate launch run_textbox.py --model=PTG --dataset=dd --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset4_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-26_12-31-16_QKVo_10/checkpoint_epoch-1 --train_batch_size=2 --accumulation_steps=48

• cross_dataset4_paper--22-23

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/jzh9mxwh?workspace=user-minji913

[minji-o-j]

QKV 학습X 버전 (tag: QKVx_#num)

• [x] 1번 서버 5

  accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_task1 --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-28_14-55-31_QKVx_1/checkpoint_epoch-3 --train_batch_size=4 --accumulation_steps=24

• cross_task1-_05-06-02
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_task-pc/runs/l7s7irry?workspace=user-minji913

• [x] 5번 서버 5 gpu 01

  accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-26_18-38-20_QKVx_5/checkpoint_epoch-2 --train_batch_size=4 --accumulation_steps=24

• cross_dataset2--48-31

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/ny9czl0d?workspace=user-minji913
  

• [x] 9번 서버 5 23

  accelerate launch run_textbox.py --model=PTG --dataset=pc --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset3_paper  --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-pc-2023-Jul-28_08-42-47_QKVx_9/checkpoint_epoch-13 --train_batch_size=4 --accumulation_steps=24

• cross_dataset3_paper--48-36

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-pc/runs/ci1ozmv2?workspace=user-minji913
  

• [x] 6번 서버 5 gpu 5,6

  accelerate launch run_textbox.py --model=PTG --dataset=dd  --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset2 --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-27_10-17-18_QKVx_6/checkpoint_epoch-10 --train_batch_size=2 --accumulation_steps=48

• cross_dataset2--13-36

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/xsnmbefv?workspace=user-minji913

• [x] 10번 서버5 gpu 2,3

  accelerate launch run_textbox.py --model=PTG --dataset=dd --model_path=facebook/bart-large --gpu_id=0,1 --find_unused_parameters=true --source_task=cross_dataset4_paper --training_option=BART-finetuning --QKV_training=False --learning_rate=3e-5 --attention_path=PTG-dd-2023-Jul-29_13-34-25_QKVx_10/checkpoint_epoch-1 --train_batch_size=2 --accumulation_steps=48

• cross_dataset4_paper--38-57
• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-dd/runs/n0khes8g?workspace=user-minji913

[minji-o-j]

• [ ] 4번 서버5 gpu0123 진행중

  accelerate launch run_textbox.py --model=PTG --dataset=xsum --model_path=facebook/bart-large --gpu_id=0,1,2,3 --find_unused_parameters=true --source_task=cross_dataset1 --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-xsum-2023-Aug-14_03-53-23_QKVo_4/checkpoint_epoch-8 --train_batch_size=2 --accumulation_steps=48 --eval_batch_size=8

• [x] 8번 서버5 gpu 4567

  accelerate launch run_textbox.py --model=PTG --dataset=xsum --model_path=facebook/bart-large --gpu_id=0,1,2,3 --find_unused_parameters=true --source_task=cross_dataset2_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-xsum-2023-Aug-14_03-53-29_QKVo_8/checkpoint_epoch-1 --train_batch_size=2 --accumulation_steps=48 --eval_batch_size=8

• cross_dataset2_paper--33-22

• https://wandb.ai/hyu-prompt-transfer/PTG-cross_dataset-xsum/runs/unv3l5o8?workspace=user-minji913

8번 prompt1 말고 accelerate launch run_textbox.py --model=PTG --dataset=xsum --model_path=facebook/bart-large --gpu_id=0,1,2,3 --find_unused_parameters=true --source_task=cross_dataset2_paper --training_option=BART-finetuning --QKV_training=True --learning_rate=3e-5 --attention_path=PTG-xsum-2023-Aug-14_03-53-29_QKVo_8/checkpoint_epoch-4 --train_batch_size=2 --accumulation_steps=48 --eval_batch_size=8