notadamking / RLTrader

A cryptocurrency trading environment using deep reinforcement learning and OpenAI's gym
https://discord.gg/ZZ7BGWh
GNU General Public License v3.0
1.73k stars 539 forks source link

Loopback window size #144

Open lukaszkn opened 3 years ago

lukaszkn commented 3 years ago

Is there loopback window specified here? Does it mean it's just 1 past value considered? Thanks

self.n_features = 6 + len(self.data_provider.columns) self.obs_shape = (1, self.n_features) self.observation_space = spaces.Box(low=0, high=1, shape=self.obs_shape, dtype=np.float16)

MichaelQuaMan commented 2 years ago

@lukaszkn I started poking around, and that's a great question.

I haven't been able to find a specific window_size setting explicit or hidden as a default. I'll keep poking around and I'll update this if I find something.

MichaelQuaMan commented 2 years ago

@lukaszkn I think you are correct:

Check out the simpler code in this project: https://github.com/AminHP/gym-anytrading

window_size is user defined and passed into __init__(), then added to self.shape in and passed to spaces.Box(... shape=self.shape)

        self.shape = (window_size, self.signal_features.shape[1])

        # spaces
        self.action_space = spaces.Discrete(len(Actions))
        self.observation_space = spaces.Box(low=-np.inf, high=np.inf, shape=self.shape, dtype=np.float32)

So, if the RLTrader code reads self.obs_shape = (1, self.n_features), then the 1 is the window_size.

Update:

in gym/spaces/Box, the shape arg becomes a tuple of 2 values, where the first is the low and the second is the high.

low in the above code becomes the window size.

gym uses these to establish the boundaries of the space.

Given the various exceptions I've seen when I try to load models, which have been train with various window sizes...

My guess is the boundaries become the model's expected observation dimensions/shape, so that if we train a model with a 25 row window size; it will expect to get 25 rows of observation data when we call the model's "predict" function.