Implementing a model that trains query and keys

[minji-o-j]

• [x] QKV_train option 수정 (True "True")
• [x] prompt testing using frozen model
• [x] prompt 저장
  • PLM을 frozen시키고 prompt 학습 -> 저장 하면 PLM과 함께 저장됨
• [x] hyperparameter setting (논문대로)
  • [x] q,k,v learning rate = 1e-3
  • [x] adam optimizer
  • [x] do not use label smoothing
  • [x] batch size X accumulated gradient = 96
  • [x] pc
  • [x] dd
  • [x] squad
  • [ ] wiki
  • [x] quora
  • [x] wp
  • [x] cnndm
  • [x] xsum
  • [ ] msn
  • [ ] mn
  • [ ] nr
  • [ ] tc
  • [x] da
  • [ ] mw

[minji-o-j]

참고 https://github.com/guoxuxu/soft-prompt-transfer/blob/a996276d7565327eaff12392b08e40b56fdb1b80/optima/mymodules/myprompt_loader.py

[minji-o-j]

참고 2 https://github.com/mkshing/Prompt-Tuning/tree/master