tcloaa / Deep-Portfolio-Theory

Autoencoder framework for portfolio selection (paper published by J. B. Heaton, N. G. Polson, J. H. Witte.)
128 stars 63 forks source link

Reverse scaling before predicting #7

Closed mattpearson closed 6 years ago

mattpearson commented 6 years ago

Greetings. Well done re-implementation of the paper. Had a quick comment/question on your use of a scalar and predicting the result. Agree that it indeed the input data should be scaled prior to estimation, however shouldn't you inverse-transform the prediction back to its original scale before you assess how it compared to the benchmark? Relevant code is below. Thank you.

    dl_scaler[s] = StandardScaler()       # Multi-layer Perceptron is sensitive to feature scaling, so it is highly recommended to scale your data
    dl_scaler[s].fit(x)
    x = dl_scaler[s].transform(x)  

    deep_learner.fit(x, y, shuffle=False, epochs=500, batch_size = 10)    # fit the model
    deep_learner.save('model/retrack_s' + str(s) + '.h5') # for validation phase use

    # is it good?
    relative_percentage = copy.deepcopy(deep_learner.predict(x))
    relative_percentage[0] = 0
    relative_percentage = (relative_percentage /100) + 1
tcloaa commented 6 years ago

Hello,

Nope, I don' think you have to do any reverse scaling for predicted y, because you are using original values of y for training.

Which means that you only perform preprocessing on input x. Let's take image classification as an example:

    mean_image = np.mean(X_train, axis = 0)
    X_train -= mean_image
    X_val -= mean_image
    X_test -= mean_image

Then you use this transformed data as your input to the model. So, train an f such that f(transformed_x) = value_y.

FYI: CS231n