search

nmheim / torsk

An echo state network (ESN) for video prediction
10 stars 5 forks source link

Imed #41

Closed nmheim closed 5 years ago

nmheim commented 5 years ago

trying to implement the least squares solution with the IMED reprojection as suggested in #37 .

what we want is:

Wout = argmin_Wout (Wout * X - Y)**2

On monday I thought we would only need to reproject the inputs/labels during the optimization phase like this:

Wout = argmin_Wout (G1/2 Wout X - G1/2 * Y)**2

so what we are actually calculating with leastSq(X, G1/2Y) is G1/2 Wout. Therefore, to get the final Wout, I am now mulitplying by the inverse of G1/2. This seems to work and is sketched here: https://github.com/nmheim/toto/blob/1e522b2daf251dd3fa3e1be2dbdd3de79e3738eb/torsk/models/numpy_esn.py#L147-L156

nmheim commented 5 years ago

By 'this seems to work' I mean, I am getting the same results for simple circle prediction (cos/sin). The mackey circle results might be a bit better. but that could also be due to randomness. have to check more thoroughly

nmheim commented 5 years ago

calc inv(G12) with inverse of eigenvalues !

nmheim commented 5 years ago

figure_1

Last layer optimized with IMED vs classic euclidean distance loss functions this is the performance on the simple circle prediction

The plot shows mean IMED(prediction, labels) of 50 runs