search

notadamking / RLTrader

A cryptocurrency trading environment using deep reinforcement learning and OpenAI's gym
https://discord.gg/ZZ7BGWh
GNU General Public License v3.0
1.73k stars 540 forks source link

Reasonable balances for trading #38

Open demirelo opened 5 years ago

demirelo commented 5 years ago

Currently, agents buy and sell using unrealistic USD and BTC balances. For instance, during a debugging session I saw one agent executing this kind of trades:

Buying BTC balance USD: 0.004812330813060336 btc held: 2.5835011089138864 Net worth: 17707.373082848768

Buying BTC balance USD: 0.0032082205420402242 btc held: 2.5835013414545522 Net worth: 17777.0501037559

It doesn't look like there is a logical lower bound on the USD and BTC that can be used in a trade. In real life one wouldn't try to (and couldn't) buy BTC with $0.0016.

Also, using correct number of decimals (i.e., 8 for BTC) is important for precision. That might be another bug actually.

silentrob commented 5 years ago

Good catch. I think there is some room for improvement on picking the position size.

arron202 commented 5 years ago

Yes, I also meet a similar issue: Bought: -9.094947017729282e-13 also during optimization on trial have the results seems overflow: Best trial: 1014384906207232.0

We may have a least trade unit like a hand in stock, if our Net worth is less than that, it should stop trading.

NervousEnergy1979 commented 5 years ago

Is the bet size determined by the amount variable or the 12 discrete action spaces? I am having a hard time understanding how the bet size is calculated, and why 12 was chosen as the size of the discrete action space.

notadamking commented 5 years ago

Great catch @xeroquark. I will gladly accept a PR fixing this issue.

@ktattan 12 discrete options allows the agent to buy using 25% increments of its balance, sell in 25% increments of its currently held BTC, or hold. There is definitely room for improvement in this action space strategy, perhaps a simple buy all/sell all/hold would perform better.