SGD on overfitted polynomials

[FelixBenning]

• true model: linear
• sample gaussian input points
• add gaussian noise to outputs

fit overparametrized polynomials. SGD should be more robust than least squares regression. Did not work so far. Figure out why