search

hugocen / freqtrade-gym

A customized gym environment for developing and comparing reinforcement learning algorithms in crypto trading.
GNU General Public License v3.0
221 stars 45 forks source link

some questions #10

Open Payback80 opened 3 years ago

Payback80 commented 3 years ago

I have tried your example and it works! Now i'd like to ask some questions for clarification

in freqtradegym.py why `obs = np.array([

row.open,

        # row.high,
        # row.low,
        # row.close,
        # row.volume,

`

are pulled out of the observation space? Why you consider at least open and close not informative? edit: At least normalized

in IndicatorforRL.py there are still the buying and selling conditions, are there for compatibility issues or for what? also because in LoadRLmodel.py the buying/selling conditions call the model.zip

i see in freqtradegym self.stake_amount = self.config['stake_amount']

self.reward_decay = 0.0005 self.not_complete_trade_decay = 0.5 self.game_loss = -0.5 self.game_win = 1.0 self.simulate_length = self.config['gym_parameters']['simulate_length']

the reward parameters, this drives me to ask: Is the profit the objective function?

in config

"gym_parameters": { "indicator_strategy": "IndicatorforRL", "fee": 0.0015, "timerange": "20201120-20201130", "simulate_length": 200 }, is timerange the observation space? If yes simulate_lenght are the Max_Episodes trained ONTO the timerange? Am i right? So regarding yours example in 15mins timeframe are 200 15min timesteps?

Do you mind switch to rllib?

``

Payback80 commented 3 years ago

Oh sorry another question, can the model.zip used with a strongly correlated pair?

hugocen commented 3 years ago

in freqtradegym.py why obs = np.array([ # row.open, # row.high, # row.low, # row.close, # row.volume,

My idea is combining with multiple data source(different type of crypto) to gain more data to simulate the trading environment in order to train a more robust agent , so I only use normalized features which can cross different markets. Maybe this isn't the right way to do so you can change these features by your desire.

in IndicatorforRL.py there are still the buying and selling conditions, are there for compatibility issues or for what?

Yes, it is for the compatibility issues.

the reward parameters, this drives me to ask: Is the profit the objective function?

Yes, of course. The win/loss condition is to make sure that each training episodes won't take too long.

is timerange the observation space?

Yes

If yes simulate_lenght are the Max_Episodes trained ONTO the timerange? Am i right? So regarding yours example in 15mins timeframe are 200 15min timesteps?

Well, not exactly. For each simulation I randomly select an entry point in the observation space. So not each episodes got exactly 200 steps.

Do you mind switch to rllib?

Sure, I will find some time to take a look into the lib and come up with an example.

can the model.zip used with a strongly correlated pair?

I can't say for sure, but you can do some paper trading to find out.

hugocen commented 3 years ago

@Payback80 I have updated an basic example for rllib. And I am working on how to load the rllib trained model.

Payback80 commented 3 years ago

ehy that's great! I've worked hard on RL in the past year, if you like we can exchange ideas

hugocen commented 3 years ago

Surer, my email is: hugo.chen731@gmail.com feel free to contact me. but I'm not an RL expert though.

Hsgngr commented 3 years ago

If yes simulate_lenght are the Max_Episodes trained ONTO the timerange? Am i right? So regarding yours example in 15mins timeframe are 200 15min timesteps?

I am still curious about this:

There are 10 days in the example config_rl.json from 2020/11/20 to 2020/11/30

10 days 24 hours * 4 (15min intervals) = 960 steps. I feel like 200 steps are really less (50 hours/ 2 days). I think 1 week might be a better select.

Well, not exactly. For each simulation I randomly select an entry point in the observation space. So not each episodes got exactly 200 steps.

Can you explain why do you enter randomly, is it because of the risk of overfitting ?

Payback80 commented 3 years ago

any advancement on rllib?

nasrin-kh commented 2 years ago

Hi When I run the following code: pip install gym The following error occurs: Installing collected packages: cloudpickle, gym ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/nasrin/freqtrade/.env/lib/python3.8/site-packages/cloudpickle' Check the permissions.

Hsgngr commented 2 years ago

Hi When I run the following code: pip install gym The following error occurs: Installing collected packages: cloudpickle, gym ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/nasrin/freqtrade/.env/lib/python3.8/site-packages/cloudpickle' Check the permissions.

run sudo pip install gym