search

vshokorov / margin_based_ensembles_boosting

Other
0 stars 1 forks source link

Обучение сетей с гэпом #3

Open vshokorov opened 2 years ago

vshokorov commented 2 years ago

image

vshokorov commented 2 years ago

Гистограммы логитов (true - max(other))

Так как занимают много места, я их скрыл. Чтобы открыть, надо нажать на текст ниже.

Click this to collapse/fold.

vshokorov commented 2 years ago

Ширина минимума, который достигается при различных значениях гэпа.

Click this to collapse/fold. Значения в конце обучения.
model Fisher trace train Fisher trace test Hessian trace train Hessian trace test
width64_gap_0.1-02-27-22:56:02 1.602e-05 1.903e-04 1.622e-05 1.948e-04
width64_gap_0.2-02-28-00:31:32 1.537e-05 2.005e-04 1.537e-05 1.919e-04
width64_gap_0.4-02-27-22:56:41 1.130e-05 1.744e-04 1.101e-05 1.775e-04
width64_gap_0.5-02-28-00:32:32 1.915e-05 1.779e-04 1.862e-05 1.796e-04
width64_gap_1.0-02-28-02:07:02 8.535e-06 1.617e-04 8.756e-06 1.616e-04
width64_gap_2.0-03-01-09:07:45 7.557e-06 1.347e-04 7.475e-06 1.330e-04
width64_gap_4.0-03-01-09:38:09 1.999e-05 1.379e-04 2.048e-05 1.353e-04
width64_gap_6.0-03-01-08:44:02 3.527e-05 1.653e-04 3.438e-05 1.657e-04
width64_gap_8.0-03-01-09:14:17 5.908e-05 1.583e-04 5.577e-05 1.613e-04

image

vshokorov commented 2 years ago
model Loss without gap Loss with gap
width64_all_DS_train 9.711e-03 9.711e-03
width64_gap_0.1_train 4.349e-03 4.658e-03
width64_gap_0.2_train 4.881e-03 5.523e-03
width64_gap_0.3_train 4.093e-03 4.917e-03
width64_gap_0.4_train 3.586e-03 4.541e-03
width64_gap_0.5_train 9.059e-03 1.162e-02
width64_gap_1.0_train 4.169e-03 6.638e-03
width64_gap_2.0_train 7.805e-03 1.411e-02
width64_gap_4.0_train 4.112e-02 8.861e-02
width64_gap_6.0_train 1.187e-01 2.643e-01
width64_gap_8.0_train 4.085e-01 9.443e-01

В следующих графиках по иксу отложено значение гэпа:

image

image image

vshokorov commented 2 years ago

VGG16_width16, CIFAR10 (fixed lr)

Click this to collapse/fold.

vshokorov commented 2 years ago

VGG16_width64, CIFAR10 (fixed lr)

Click this to collapse/fold.

ResNet9, CIFAR10

Click this to collapse/fold.

ResNet18, CIFAR10

Click this to collapse/fold.

VGG16_width16, VGG16_width64, ResNet9, CIFAR10

Click this to collapse/fold.

vshokorov commented 2 years ago

ResNet18, CIFAR100

Click this to collapse/fold.

ResNet9, CIFAR100

Click this to collapse/fold.

VGG16_width64, WIdeResNet, ResNet9, CIFAR100

Click this to collapse/fold.
ГРАФИКИ ДЛЯ ResNet9 НЕ ПОПРАВИЛ!!!!

vshokorov commented 2 years ago

ResNet9, with BN CIFAR100

Click this to collapse/fold.

vshokorov commented 2 years ago

ResNet9, SVHN

Click this to collapse/fold.

ResNet18, SVHN

Click this to collapse/fold.

VGG16_width64, SVHN (fixed lr)

Click this to collapse/fold.

ResNet9, VGG16_width64, SVHN

Click this to collapse/fold.

vshokorov commented 2 years ago

Зависимость нормы градиента при обучении width64 SVHN ADAM

Click this to collapse/fold.

vshokorov commented 2 years ago

Будем делать гэп 5 на каком-то проценте плохих объектов. У остальных гэп равен нулю.

Click this to collapse/fold.
Легенда некорректна!!

vshokorov commented 2 years ago

Будем делать гэп 5 на каком-то проценте хороших объектов. У остальных гэп равен нулю.

Click this to collapse/fold.
Легенда некорректна!!