other condition
dropout=0.1
Learnable weight of averaging hidden layer, n_hid=12, average, learn=False
implement consideration of text_areas
implement remove_excessive_padding
[No use]train only positive and negative
label smoothing 0.05
lr 1e-5
different learning rate (x30)
Sentiment attention head
mode (n_element, reduction, dropout, add) : score
Model_v1_9_0 (5, 4, 0.1, None) : cv 0.546743 (only posi and nega)
Model_v1_9_2 (3, 4, 0.0, None) : cv 0.547488 (only posi and nega) Model_v1_9_1 (5, 4, 0.0, None) : cv 0.551155 (only posi and nega) Model_v1_9_3 (8, 4, 0.0, None) : cv 0.551247 (only posi and nega) Model_v1_9_4 (16, 4, 0.0, None) : cv 0.545075 (only posi and nega) Model_v1_9_5 (32, 4, 0.0, None) : cv 0.550671 (only posi and nega) Model_v1_9_6 (64, 4, 0.0, None) : cv 0.551484 (only posi and nega)
Model_v1_9_7 (8, 4, 0.0, linear_head) : cv 0.552004 (only posi and nega) Model_v1_9_8 (8, 4, 0.0, linear_head) : cv 723900
other condition dropout=0.1 Learnable weight of averaging hidden layer, n_hid=12, average, learn=False implement consideration of text_areas implement remove_excessive_padding [No use]train only positive and negative label smoothing 0.05 lr 1e-5 different learning rate (x30)