Regarding Reward and Profit

[AlphaNumeric99]

Hi,

I recently used anytrading with a custom environment and stable baseline's PPO2 algo.

After running the evaluation part 10 times my output was something like

info {'total_reward': 24392200.00000009, 'total_profit': 0.9844407417070604, 'position': 0}
info {'total_reward': 48881799.99999967, 'total_profit': 1.011612620710015, 'position': 0}
info {'total_reward': 51085300.00000165, 'total_profit': 1.013074701451891, 'position': 1}
info {'total_reward': 14793399.999999177, 'total_profit': 0.9767670021357563, 'position': 0}
info {'total_reward': 17957400.000001136, 'total_profit': 0.9815584135159401, 'position': 0}
info {'total_reward': -2354400.0000011073, 'total_profit': 0.9607471236716814, 'position': 1}
info {'total_reward': 20103799.9999998, 'total_profit': 0.9839828662099608, 'position': 0}
info {'total_reward': 19209400.000002127, 'total_profit': 0.9826626717429163, 'position': 1}
info {'total_reward': 14625800.00000124, 'total_profit': 0.9773373249065562, 'position': 1}
info {'total_reward': 53867999.99999998, 'total_profit': 1.0180095847348958, 'position': 1} 

As far as i understand profit, if profit is >1 it is profit otherwise <1 is loss. My question is why is the total_reward positive in cases where the total_profit is actually <1 (loss).

Also it seems like it trades too frequently even when it shouldn't, Can we add one such action like wait to see a bigger price difference or trend. (Sorry if there's a proper term for it, I am new to trading stuff)

[kmafutah]

Am not sure if this would help. But just assumed that maybe the negative profits where from just shorts... try with sine wave... Rgds KM

Sent from my iPhone

On 29 Apr 2021, at 11:15, AlphaNumeric99 @.***> wrote:

 Hi,

I recently used anytrading with a custom environment and stable baseline's PPO2 algo.

After running the evaluation part 10 times my output was something like

info {'total_reward': 24392200.00000009, 'total_profit': 0.9844407417070604, 'position': 0} info {'total_reward': 48881799.99999967, 'total_profit': 1.011612620710015, 'position': 0} info {'total_reward': 51085300.00000165, 'total_profit': 1.013074701451891, 'position': 1} info {'total_reward': 14793399.999999177, 'total_profit': 0.9767670021357563, 'position': 0} info {'total_reward': 17957400.000001136, 'total_profit': 0.9815584135159401, 'position': 0} info {'total_reward': -2354400.0000011073, 'total_profit': 0.9607471236716814, 'position': 1} info {'total_reward': 20103799.9999998, 'total_profit': 0.9839828662099608, 'position': 0} info {'total_reward': 19209400.000002127, 'total_profit': 0.9826626717429163, 'position': 1} info {'total_reward': 14625800.00000124, 'total_profit': 0.9773373249065562, 'position': 1} info {'total_reward': 53867999.99999998, 'total_profit': 1.0180095847348958, 'position': 1} As far as i understand profit, if profit is >1 it is profit otherwise <1 is loss. My question is why is the total_reward positive in cases where the total_profit is actually <1 (loss).

Also it seems like it trades too frequently even when it shouldn't, Can we add one such action like wait to see a bigger price difference or trend. (Sorry if there's a proper term for it, I am new to trading stuff)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[AlphaNumeric99]

I don't understand how to implement sine wave?

[kmafutah]

I found this to work shame it’s on baseline and not keras and plaidml

https://stackoverflow.com/questions/64687298/stable-baselines-dqn-not-performing-properly

... the solution for your last question is in the question in the stack overflow.

Hope it helps. :)

If I may ask my good people has anyone come across a gym-anytrading environment with Buy, Sell,Hold,Exit actions ? Or it’s there and I don’t see it?

Rgds KM

On Sun, 2 May 2021 at 22:31, AlphaNumeric99 @.***> wrote:

I don't understand how to implement sine wave?

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/AminHP/gym-anytrading/issues/35#issuecomment-830867374, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7JXS7Q6GS2VGQLMGMN5LTTLWZCLANCNFSM43ZPUWSA .

--

Kudakwashe Mafutah [image: https://]about.me/kmafutah https://about.me/kmafutah?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=plaintext

[AminHP]

Hi @AlphaNumeric99

About the first question, this explanation might help.

And the second question, there is no need to add a new action (however it is possible). You can manage the trading rate by adding some punishment/reward strategies.

[AlphaNumeric99]

1. Understood. Hopefully you update here if you find an even better reward strategy.
2. Thank you