Open Michaeljurado42 opened 1 month ago
Describe the bug
I was converting lava-dl networks to lava via netx and noticed that the weight values were inflated. From what I understand, loihi2 should optimize weight matrices.
The issue can be solved by calling optimize_weight_bits after truncate_weights.
To reproduce current behavior
Try running this test: https://github.com/lava-nc/lava-dl/files/15328518/small_test.zip and you will see that the weights are scaled from 64 to 128 with weight_exp = -1.
Expected behavior weights should be optimized to 1 with weight_exp = 6.
Additional context
This issue can also technically be fixed in lava-dl by pre-optimizing the weights in lava-dl like this.
Although, if we properly fix the issue in lava we can remove most of the logic in lava-dl's fix_optimize_weights which is redundant with lava anyway
Describe the bug
I was converting lava-dl networks to lava via netx and noticed that the weight values were inflated. From what I understand, loihi2 should optimize weight matrices.
The issue can be solved by calling optimize_weight_bits after truncate_weights.
To reproduce current behavior
Try running this test: https://github.com/lava-nc/lava-dl/files/15328518/small_test.zip and you will see that the weights are scaled from 64 to 128 with weight_exp = -1.
Expected behavior weights should be optimized to 1 with weight_exp = 6.
Additional context
This issue can also technically be fixed in lava-dl by pre-optimizing the weights in lava-dl like this.
Although, if we properly fix the issue in lava we can remove most of the logic in lava-dl's fix_optimize_weights which is redundant with lava anyway