finetuing BART model - Githubissues

ailab-prompt-transfer / TextBox

Implement of PTG

https://github.com/RUCAIBox/TextBox

MIT License

0 stars 0 forks source link

finetuing BART model #13

Closed minji-o-j closed 1 year ago

minji-o-j commented 1 year ago

[x] 모델 불러오기
- [x] path
- [x] path error
- [x] QKV o 버전
- [x] QKV x 버전
[x] QKV, query key 학습 X
[x] BART 모델 학습
[x] BART 모델 저장
[x] arg연결
[x] test 실험
[x] hyperparameter bart맞춤
- learning_rate : 3e-5

minji-o-j commented 1 year ago

        "args": ["--model=PTG",
                "--dataset=pc",
                "--model_path=facebook/bart-large",
                "--gpu_id=0,1",
                "--find_unused_parameters=true",
                "--source_task=cross_task1",
                "--training_option=BART-finetuning", //adaptive-attention",
                "--QKV_training=True",
                "--attention_path=PTG-pc-2023-Jul-26_05-16-58_QKVo_1/"--attention_path=PTG-pc-2023-Jul-26_05-16-58_QKVo_1/checkpoint_epoch-11" ,
                "--learning_rate=3e-5"