API for GridWorlds.jl environments

[Sid-Bhatia-0]

This is to decide upon a consistent API that would be implemented by GridWorlds environments.

I propose to use the ReinforcementLearningBase interface (see here) for this package for following reasons:

1. The RLBase interface is quite general and it covers several of the typical RL scenarios, including but not limited to multiple agents, different stochasticity styles, sequential/simultaneous moves etc... My hunch is that GridWorlds will soon expand and require such sophisticated features (say multi-agent, for example for a gridworld-soccer like environment).

2. It is well integrated with the rest of the JuliaRL ecosystem. RLCore utilizes RLBase and implements a lot of commonly required functionalities for RL tasks. RLZoo utilizes RLBase and implements several useful algorithms that can be used off the shelf. We can utilize all of that work for free.

3. If we choose to implement an independent API from scratch specifically tailored for GridWorlds, it is possible that we will encounter some conversion friction while trying to convert it to RLBase interface in order to work with the rest of the JuliaRL packages. In this case, we would mostly need to write our own implementations of the functionalities offered in RLCore or RLZoo. On the other hand, if we make a GridWorlds API from scratch that is as general as the RLBase one, such that we never encounter any lossy conversion, then well, we might as well have used RLBase API in the first place.

4. In the rare case, if we require something that is not currently offered by RLBase (our current environments are already well covered I think), then this an improvement opportunity for RLBase, and RLBase interfaces would expand accordingly. In the case that it does not happen for whatever reason, we can always expand our own API for such special cases while sticking to RLBase interface as much as possible. Of course, since we would be incorporating features never implemented in the rest of the JuliaRL ecosystem, we would have to consequently write our own implementations for such special features. But this is a rare case, I think. And moreover, if we are to not follow the RLBase API, then we anyways have to write custom implementations without much choice.

Of course, this would add a dependency upon RLBase. But I feel that is very much worth it.

I would love to hear other peoples' thoughts on this.

[Sid-Bhatia-0]

RLBase API has been added to all current environments.