notadamking / RLTrader

A cryptocurrency trading environment using deep reinforcement learning and OpenAI's gym
https://discord.gg/ZZ7BGWh
GNU General Public License v3.0
1.73k stars 540 forks source link

Live trading implementation #21

Open PromediaB opened 5 years ago

PromediaB commented 5 years ago

Want to ask you if any one already integrated the code with any exchanger, as im planning to link it today or tomorrow with Bitmex to start testing how its will done.

I need to know if any one already done it? if yes how is the result ?

There is any proposal ?

silentrob commented 5 years ago

I think the next logical step would be to build out a paper trader. Something that fetches market data and mocks a buy/sell and takes into account commission. The system needs a little more logging and debugging before doing any actual trading (IMHO).

samfultonjr commented 5 years ago

@PromediaB Have you implemented a live trader yet? If so, please fork from the repo and share the project. :)

silentrob commented 5 years ago

Not yet. I've built a NodeJS trading bot, and considered wrapping this in a simple flask server to handle the predictions, but that I don't think this project is ready for that just yet. And it might make more sense to just re-implement the paper trader in this project... seems like that is where this is going anyway. (If the model produces great results)

samfultonjr commented 5 years ago

Yeah, as a node dev I would much rather see this in node. I know tensorflow js allows you to convert a python model into a js model ( https://github.com/tensorflow/tfjs-converter ), so if I can get my hands on the model I would love to make a trader using node js and the acclaimed 850% returns model. @silentrob

silentrob commented 5 years ago

Oh, ya! I would be totally open to that approach too!

samfultonjr commented 5 years ago

Perfect, I know notAdamKing posted the best models in another thread,

Sortino 1: https://www.dropbox.com/s/lqjf7pqxd7il38h/ppo2_sortino_2.pkl?dl=0 Sortino 2: https://www.dropbox.com/s/dkvre43zikagfik/ppo2_sortino_3.pkl?dl=0

Profit 1: https://www.dropbox.com/s/a9diw09vuce5v4c/ppo2_profit_4.pkl?dl=0 Profit 2: https://www.dropbox.com/s/zjc9rvgdgx3gul6/ppo2_profit_2.pkl?dl=0

I have never used pkl before and I thought they where just saved tensorflow models, but when I tried to load them up it said it was missing modules from this repo. I am not sure what is in them since I haven't looked at exactly how this thing is working, but I know how to work with tensorflow models. Do you know how to get the tensorflow models out? @silentrob

silentrob commented 5 years ago

So, I'm in a holding pattern waiting on #28 If we can find and re-train a good model I will explore exporting it to tensorflow.js

notadamking commented 5 years ago

I will be completing this tonight, along with the following requirements:

maxmatical commented 5 years ago

I am trying implement something that would allow new observations in the live trading environment.

The process would be (within the environment):

  1. Grab live data
  2. Append it to existing dataframe of all pricing data (for scaling)
  3. Create a new environment updated with info on
    • current cash balance
    • current btcs held
    • current btc worth
  4. reset observations (needed to make the new environment run) but don't reset state
  5. start at the last row and run one step

the current issue is self.current_step always resets to 0. I should be able to add a self.live or some indicator so that when it's true, then self.current_step defaults to the last step, then when you step through the environment, it just performs the last action.

Can anyone see any issues with this approach?

Usmaniatech commented 5 years ago

@maxmatical have u achieved what you have described above ?