issues
search
accosmin
/
nano
C++ library [machine learning & numerical optimization] - superseeded by libnano
MIT License
1
stars
0
forks
source link
Auto-tune the minibatch size
#163
Closed
accosmin
closed
7 years ago
accosmin
commented
7 years ago
the minibatch size has a great influence on performance and it depends on the task at size
tune 2 parameters: batch size {8
T, 16
T, 32
T, 64
T where T = logical cpus} & factor 1 + {0, 1e-2, 1e-3, 1e-4}
another option is to tune a single parameter: keep the batch size as a parameter, but tune the factor {0, 1e-4, ...., 1e-2}
accosmin
commented
7 years ago
solution:
modify stoch_solver_t to have ::tune() & ::minimize()
modify stoch_solver_t to store (tuned) parameters as configurable_t::config()
rename solver files to solver_xxx.cpp/h
also rename layers to layer_xxx.cpp/h
move the content of stoch_loop t as private utilities in stoch_solver_t
modify stoch_trainer_t to tune the minibatch factor by calling ::tune(factors...) in a loop