However, since the sensorium model sets core_bias=False per default - which was ignored and behaved like core_bias=True because independent_bn_bias=True - now core_bias=True this should be the new default to (a) make that argument work in the first place and (b) keep the same behavior as with the neural predictors version from back then.
Because
independent_bn_bias
was per defaultTrue
: https://github.com/ecker-lab/neuralpredictors/blob/7802bed1fda9b6536ff7bf94326fb45a50cfa1d3/neuralpredictors/layers/cores/conv2d.py#L55the
bias
parameter was ignored: https://github.com/ecker-lab/neuralpredictors/blob/7802bed1fda9b6536ff7bf94326fb45a50cfa1d3/neuralpredictors/layers/cores/conv2d.py#L208-L225 . Overall, the default was to add scale and bias to the batch norm: https://github.com/ecker-lab/neuralpredictors/blob/7802bed1fda9b6536ff7bf94326fb45a50cfa1d3/neuralpredictors/layers/cores/conv2d.py#L211We removed
independent_bn_bias
in https://github.com/sinzlab/neuralpredictors/pull/221 . The default behavior of the core remained unchanged (batch_norm_scale and bias default to true): https://github.com/ecker-lab/neuralpredictors/blob/e354212e227793ef3e2d465739854e0f154f01b5/neuralpredictors/layers/cores/conv2d.py#L61 and https://github.com/ecker-lab/neuralpredictors/blob/e354212e227793ef3e2d465739854e0f154f01b5/neuralpredictors/layers/cores/conv2d.py#L56However, since the sensorium model sets
core_bias=False
per default - which was ignored and behaved likecore_bias=True
becauseindependent_bn_bias=True
- nowcore_bias=True
this should be the new default to (a) make that argument work in the first place and (b) keep the same behavior as with the neural predictors version from back then.