Closed stabilus closed 3 years ago
To have a continuous action space, we need to rewrite all the three environments. Personally, I like this idea and I am up to implement continuous_trading_env
, continuous_forex_env
, and continuous_stocks_env
. But, I'm not sure if I have enough time for that at this moment.
I was thinking about having a range between -1 and +1 for action values. The range -1 to 0 indicates a sell signal while the range 0 to +1 indicates a buy signal. Also, the absolute value of that can represent the percentage of the networth and multiply self._total_profit
for the next trade. What do you think?
It would probably be simpler if you just used 0-1 and let the algo decide whether that means buying or selling. i.e. if you're invested 50%, an action of 0.3 would imply selling the position from a percentage of 50% down to 30%.
Hello @AminHP ,
First of all, great work on this project as it is proving to be very useful to me.
I also prefer the [-1, +1] range for the continuous case as it usually the most common in the research community.
On a similar note, for the discrete case, how do you recommend extending the TradingEnv class to define my action values?
Thanks in advance =)
Hi @xicocaio, I'm glad it's been useful to you :)
Thanks for the information. I will consider it.
I don't exactly understand your question. Do you want to extend the TradingEnv
to support more than just two actions? For example, you want to have 4 discrete actions, right?
Hi,
My issue is that I want to extend TradingEnv,
but I want to use other values. Or even add other actions, as such:
class Actions(Enum):
Sell = -1
Hold = 0
Buy = 1
Then, because these actions are in the base class, I found it not very practical to extend them in a subclass.
So my question is about what do you think is the best way I could deal with that.
Tks!
@xicocaio Yes you are right, this is a fundamental change and affects almost all parts of the trading environments (both logic and rendering). So, I think it is better to fork and rewrite those parts according to your custom actions.
Ok, I will do that, Thank you!
class Actions(Enum): Sell = -1 Hold = 0 Buy = 1
@xicocaio I was using this format, but my model just holds. Something similar happened with you?
@luccosta Well, it depends on how do you want your env and model to work.
In my case, I had to follow a specific design of a specific scientific paper, and so I had to fork this repo to make adjustments here (changes are at the feat-moody_env branch).
In summary, when adopting continuous actions, you have to take into account what @AminHP said:
I was thinking about having a range between -1 and +1 for action values. The range -1 to 0 indicates a sell signal while the range 0 to +1 indicates a buy signal. Also, the absolute value of that can represent the percentage of the networth and multiply self._total_profit for the next trade.
Also, you have to think about how these changes impact how you calculate the return (additive or multiplicative) as described in the mentioned paper.
I just released a new project that supports the required feature.
Thanks will be trying it out this weekend and will provide feedback. Once again thanks
Rgds KM
Sent from my iPhone
On 7 Sep 2021, at 21:50, Mohammad Amin Haghpanah @.***> wrote:
I just released a new project that supports the required feature.
https://github.com/AminHP/gym-mtsim
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.
say I wanted to train actions to be in a range between 0 and 1, whereby the number represents the percentage of the networth I should be invested in the asset. The resulting action is then to buy or sell the difference.