huseinzol05 / Stock-Prediction-Models

Gathers machine learning and deep learning models for Stock forecasting including trading bots and simulations
Apache License 2.0
7.96k stars 2.81k forks source link

How to include more features during real-time training? #57

Closed windowshopr closed 5 years ago

windowshopr commented 5 years ago

First off, I am absolutely loving your projects. I'm having a lot of fun training models and utilizing the different programs you have set up. Great work!

I'm wondering now, in the real-time agent, how would we go about adding more than just Close and Volume features to the training model? In the "realtime-evolution-strategy.ipynb", I've changed all of the "parameters" variables to say:

parameters = [df['Close'].tolist(), df['Volume'].tolist(), df['Open'].tolist()]

...to even just include the Open price as well, although I'd want to include High and Low as well, among other features in my dataframe. When I run the code, I get a traceback error that says:

training stock AMD.csv
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-60220d8677a3> in <module>
     17                       initial_money = initial_money,
     18                       real_trend = real_trend,
---> 19                       minmax = minmax)
     20     else:
     21         agent.change_data(timeseries = scaled_parameters,

<ipython-input-9-d96bbd4dcc7e> in __init__(self, model, timeseries, skip, initial_money, real_trend, minmax)
     19         )
     20         self.minmax = minmax
---> 21         self._initiate()
     22 
     23     def _initiate(self):

<ipython-input-9-d96bbd4dcc7e> in _initiate(self)
     29         self._capital = self.initial_money
     30         self._queue = []
---> 31         self._scaled_capital = self.minmax.transform([[self._capital, 2]])[0, 0]
     32 
     33     def reset_capital(self, capital):

~\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\preprocessing\data.py in transform(self, X)
    387                         force_all_finite="allow-nan")
    388 
--> 389         X *= self.scale_
    390         X += self.min_
    391         return X

ValueError: operands could not be broadcast together with shapes (1,2) (3,) (1,2) 

I'm not super familiar with what's happening with this error, so I'm wondering if you (or someone) would be able to shed some light on it? The "input_size" changes from the default 79 to 117 if that matters? Thanks! Love the work you've done!

huseinzol05 commented 5 years ago

I will improve realtime agent to make proper documentations and flow code.

windowshopr commented 5 years ago

That would be fantastic! Thank you for all your work. I’m enjoying its functionality already!

ycote88 commented 4 years ago

Any update on this, I still cannot find how to add more feature ?

IISuperluminaLII commented 4 years ago

when initing minmax(feature_range.... either remove it or give it higher range

billbarni commented 4 years ago

I already increased a LOT the range and it does not work, even with very small data variation.

At: self._scaled_capital = self.minmax.transform([[self._capital, 2]])[0, 0]

I get: ' ValueError: operands could not be broadcast together with shapes (1,2) (4,) (1,2) '

I understand that this 'self._scaled_capital = self.minmax.transform([[self._capital, 2]])[0, 0]' will get a 'scaled capital', but the array that was scaled is of a different size... I then shimmed another MinMaxTransformer that cares only about the 'Close' data... But only increasing the number of parameters will do anything?

I see strange lines like this:

def trade(self, data):
        """
        you need to make sure the data is [close, volume]
        """
        scaled_data = self.minmax.transform([data])[0]
        real_close = data[0]
        close = scaled_data[0]
        if len(self._queue) >= window_size:
            self._queue.pop(0)
        self._queue.append(scaled_data)

Are you cutting out ALL the data and considering only price and volume back again here?