Open zsunberg opened 1 year ago
Thanks for kicking off this discussion! What do you mean by use CommonRL directly? I've been wondering for some time whether we should use consistent naming with CommonRL. I haven't gone through every method in CommonRL, so I'm not willing to commit to the exact naming 100%, but I would love to converge on one set of terms & apis. My thought would be that we first do this for the methods which are already included in CommonRL, then in a second step look into env's. Thoughts? @HenriDeh
Concretely, I mean things like https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/blob/abae05c358635a29c9e7c0311e3db5066d5de724/src/ReinforcementLearningBase/src/CommonRLInterface.jl#L51 where I'd be happy to use valid_actions
and drop legal_action_space
What do you mean by use CommonRL directly?
By this option, I mean completely deprecating and then removing RLCore.AbstractEnv
and using CommonRL.AbstractEnv
and the methods from CommonRL
everywhere within RL.jl. This would be a big change, and I don't understand all the consequences yet.
Thoughts? @HenriDeh
I am so in favor of this. I don't think it would be that overwhelming of a change. Deprecating first, then dropping is a good idea because many algorithms are not tested at the moment.
Hi everyone,
It has been cool to see the recent flurry of contributions to this package, especially by @jeremiahpslewis. In a recent discussion, someone asked what would facilitate cooperation between the POMDPs.jl and JuliaRL communities. I was thinking about this a bit more and came to the conclusion:
Separating out the environment interface would be the most helpful change for expanding collaboration.
There are a few reasons for this:
If the environment is separated out (and is sufficiently flexible), I would probably convert some important packages like MCTS and POMCP to use it. Then, they could be much more compatible with RL.jl.
A final note: In principle, CommonRLInterface could be a candidate for a separated-out environment interface, but I do not think it can be successful unless RL.jl chooses to use it directly. To be clear, I would vigorously advocate for this, and I am happy to discuss why, but I recognize that this would be biased since I wrote most of that package.