search

lava-nc / lava-dl

Deep Learning library for Lava
https://lava-nc.org
BSD 3-Clause "New" or "Revised" License
149 stars 71 forks source link

Allow slayer norms to use parameters #301

Closed Gavinator98 closed 3 months ago

Gavinator98 commented 4 months ago

User story

As a user, I want to modify the arguments to slayer neuron norms. The neuron norms provide arguments to adjust momentum and other parameters: https://github.com/lava-nc/lava-dl/blob/543efaac136bfb028f1445f0384fe36555c0fc21/src/lava/lib/dl/slayer/neuron/norm.py#L36

However, when used the neuron calls the constructor with no arguments: https://github.com/lava-nc/lava-dl/blob/543efaac136bfb028f1445f0384fe36555c0fc21/src/lava/lib/dl/slayer/neuron/base.py#L146

As a workaround it might be possible to use a lambda function as the norm argument:

params = {
    "norm": lambda x: slayer.neuron.norm.MeanOnlyBatchNorm(momentum=0.2)
}
PhilippPlank commented 4 months ago

Thank you for this suggestion. @bamsumit Could you take a look, when you find some time :)

bamsumit commented 3 months ago

@Gavinator98 The intended usage pattern is to call a lambda like you mention. This is by design because the norm could have other signatures and therefore cannot be known in advance.