Strategy #2: Random Local Search
The "random perturbation" of the form \delta W which is added to W doesn't make sense - the formula describes scaling W by
(1+ \detla) instead of adding to W a random (independent from W), weight matrix. You can see that the code updates W correctly:
W + np.random.randn(10, 3073) * step_size
e.i. a random matrix times \delta := step_size is added to W, instead of adding \delta times W to W.
I proposed in #232 to change \delta W to \delta W'.
232
Strategy #2: Random Local Search The "random perturbation" of the form \delta W which is added to W doesn't make sense - the formula describes scaling W by (1+ \detla) instead of adding to W a random (independent from W), weight matrix. You can see that the code updates W correctly:![selected-200720-1804-14](https://user-images.githubusercontent.com/49984106/87953540-d897b880-cab3-11ea-93c6-2e08886f3e51.png)
W + np.random.randn(10, 3073) * step_size
e.i. a random matrix times \delta := step_size is added to W, instead of adding \delta times W to W. I proposed in #232 to change \delta W to \delta W'.