bquast / rnn

Recurrent Neural Networks in R
https://qua.st/rnn
73 stars 28 forks source link

predict_lstm and clean_lstm functions not found? #15

Closed albert3858 closed 7 years ago

albert3858 commented 7 years ago

LSTM version of trainr works without issues and produces a model. But attempting to run predict_lstm and clean_lstm functions results in "could not find function" errors in both cases.

bquast commented 7 years ago

@DimitriF

DimitriF commented 7 years ago

The function predict_lstm and clean_lstm are not exported, they are called from predictr and trainr respectively. In the model list returned during the training, there is an object stating if the training was with classic rnn or the experimental lstm or gru. So you just have to use the predictr function. I didn't manage to make the lstm learn anything yet, something is wrong somewhere in the code so let us know if it is learning.

albert3858 commented 7 years ago

Duh! Apologies. Will test and report back.

albert3858 commented 7 years ago

Well something seems to be happening. Compared with a DNN with weights initialised by stacked autoencoder it does much the same. Confusion matrix and statistics below based on out of sample data. The problem was prediction of hourly FX rates three hours ahead with only very naive inputs.

Confusion Matrix and Statistics

      Reference

Prediction 0 1 0 1641 1167 1 1225 1701

           Accuracy : 0.5828          
             95% CI : (0.5699, 0.5956)
No Information Rate : 0.5002          
P-Value [Acc > NIR] : <2e-16          

              Kappa : 0.1657          

Mcnemar's Test P-Value : 0.2438

        Sensitivity : 0.5726          
        Specificity : 0.5931          
     Pos Pred Value : 0.5844          
     Neg Pred Value : 0.5813          
         Prevalence : 0.4998          
     Detection Rate : 0.2862          

Detection Prevalence : 0.4897
Balanced Accuracy : 0.5828

   'Positive' Class : 0 
albert3858 commented 7 years ago

|For comparison, here are the confusion matrix and stats for the DNN+SAE:

Confusion Matrix and Statistics

      Reference

Prediction 0 1 0 1661 1172 1 1205 1696

           Accuracy : 0.5855          
             95% CI : (0.5726, 0.5982)
No Information Rate : 0.5002          
P-Value [Acc > NIR] : <2e-16          

              Kappa : 0.1709          

Mcnemar's Test P-Value : 0.5116

        Sensitivity : 0.5796          
        Specificity : 0.5914          
     Pos Pred Value : 0.5863          
     Neg Pred Value : 0.5846          
         Prevalence : 0.4998          
     Detection Rate : 0.2897          

Detection Prevalence : 0.4941
Balanced Accuracy : 0.5855

   'Positive' Class : 0               
DimitriF commented 7 years ago

Result looks the same for both indeed, Just a question though, if it is a rate, shouldn't you have a numeric value instead of a categorical ?

albert3858 commented 7 years ago

Sorry good point, it's actually predicting whether the closing price three bars (hours) ahead is higher or lower than the current price, not the actual value. Hence, as you spotted, the categorical.