moraieu / query-selector

LONG-TERM SERIES FORECASTING WITH QUERYSELECTOR – EFFICIENT MODEL OF SPARSEATTENTION
Apache License 2.0
75 stars 18 forks source link

InverseTransform only on prediction, to avoid nan loss #8

Closed simoneliasen closed 1 year ago

simoneliasen commented 1 year ago

Hi again, and thanks for the implementation of the 'ms' feature.

Two questions:

  1. Is it possible to only inversely transform the numbers of a prediction? Running inverse transform or disabling scaling on an entire training explodes the gradient with no effort. In short; is it possible to only do an inverse transformation on the pred_len?

  2. With 'ms' enabled, how would one expand on it to predict three outputs instead of a singular, is it as simple as feeding a list to the target, which is now defined as 'OT' ?

I hope you can help out, best regards, and again thanks for your contribution!

simoneliasen commented 1 year ago

Possible answer to question 1:

For other curious people, the inverse transformation can be done exclusively on the validation /prediction set by, adding this in the StandardScaler().transform method in tools.py.

np.save('std.npy', std ) np.save('mean.npy', mean)

and then later, once you want a value inversely transformed any place, do:

s= np.load('std.npy', std ) m = np.load('mean.npy', mean)

inverted_data = data * s + m

I am still looking for a way to expand 'ms' to do multiple outputs (3 in my case)

simoneliasen commented 1 year ago

Possible answer to question 2:

Going to dataloader.py, data_y is to my understanding the target-data, so instead of:

self.data_y = self.data_y[:, [target_index]]

Which, serves a prediction on a singular target. We can instead slice data_y to get predictions to the first 3 columns in our dataset:

self.data_y = self.data_y[:, 0:3]

I'll let this issue stay open, to get confirmation that my observations are true