DeiT Got 71.7 Macri-F1 Score

[jinmang2]

제 configuration을 script하기 전까지 issue에서 관리하도록 하겠습니다...ㅎㅎ

2nd trial 현재 리더보드 4위

[jinmang2]

8월 23일 월요일 1st trial

• model: DeiT

args = TrainingArguments(
    output_dir="output",
    do_train=True,
    do_eval=True,
    evaluation_strategy="steps",
    learning_rate=2e-5,
    per_device_train_batch_size=128,
    per_device_eval_batch_size=128,
    num_train_epochs=10,
    eval_steps=50,
    save_steps=50,
    save_total_limit=10,
    weight_decay=0.01,
    load_best_model_at_end=True,
    metric_for_best_model="f1",
    logging_dir='logs',
    report_to="wandb",
    run_name="210823-trial1"
)

[jinmang2]

Trial 2 -> best

• https://huggingface.co/jinmang2/deit-face-mask-2
• seed 42, 80%

  args = TrainingArguments(
  output_dir="output", 
  do_train=True, 
  do_eval=True, 
  evaluation_strategy="steps", 
  learning_rate=3e-5, 
  per_device_train_batch_size=128, 
  per_device_eval_batch_size=128, 
  num_train_epochs=5, 
  eval_steps=50, 
  save_steps=50, 
  save_total_limit=10, 
  weight_decay=0.01, 
  warmup_steps=200, 
  load_best_model_at_end=True, 
  metric_for_best_model="f1",
  logging_dir='logs', 
  report_to="wandb", 
  run_name="210823-trial2" 
  )

[jinmang2]

7, 8 -> label smoothing 추가 1, 2 -> 기본 세팅 + 80% 활용 3~8 -> 90% 활용, 모델은 동일하지만 configuration을 조금씩 다르게 + transform 추가 ... 그러나 그닥 효과 X

• transform이 하나도 적용이 안된 상태였다!!! ㅎ.... 어쩐지....

7	Finished	75.0952%	0.6661	2021-08-24 04:50
6	Finished	74.2063%	0.6684	2021-08-24 03:37
5	Finished	73.8254%	0.6421	2021-08-24 01:18
4	Finished	72.2222%	0.6370	2021-08-24 00:13
3	Finished	76.2063%	0.6862	2021-08-23 22:46
2	Finished	78.3175%	0.7173	2021-08-23 19:24
1	Finished	76.3175%	0.6844	2021-08-23 18:33