[Dev] 추출 + 생성모델 Ensemble 개발 노트

[Amber-Chaeeunk]

work with @hrxorxm

Soft

• [x] ensemble model

  • settings in ModelingArguments
  • architectures : BartForExtractionGenerationEnsemble
  • reader_type : ensemble

• [x] ensemble preprocessing

  • setting in TokenizerArguments
  • max_seq_length: 1000
  • max_label_length: 128
  • setting in ModelArguments
  • reader_type: ensemble
  • 특이사항: truncation시 overflowing samples는 날림

• [x] Ensemble trainer

  • trainer : QuestionAnsweringEnsembleTrainer
  • data collator : DataCollatorForSeq2Seq
  • setting in Seq2SeqTrainingArguments
    • predict_with_generate: False
    • generation_max_length: null #설정
    • generation_num_beams: null #설정

Hard

• [ ] Judgment algorithm (폐기)