After ~ > 100 iterations, weight values for dummy weights appear to start updating. When eliminating the weight update steps, the weights do not change indicating that the corruption occurs in the weight update step.
Objective
[ ] fix weight corruption.
[ ] add in tests for weight corruption if possible.
Description
After ~ > 100 iterations, weight values for dummy weights appear to start updating. When eliminating the weight update steps, the weights do not change indicating that the corruption occurs in the weight update step.
Objective
References