Question: Mapping discrete actions to continuous space

[KandBM]

I've trained a PPO using 20 discrete actions (controlling electrical SOC) and I'm trying to explain what each action does, is there a way to map them back to the continuous space from [-1.0, 1.0]? I'm assuming 0 is a full charge and equivalent to 1.0, and 19 to -1.0, but how could I map out the intermediate values?

Thanks!

[kingsleynweye]

@KandBM, you can create your own custom wrapper that helps you with the mapping. The idea behind wrappers in explained in the Gym docs.

Alternatively, you may be able to use the in-built DiscreteObservationWrapper and DiscreteActionWrapper wrappers. You can also use the DiscreteSpaceWrapper wrapper that combines the two. These wrappers were written for the purpose of being able to use a Tabular Q-Learning agent in the environment but might work for your PPO implementation. If you want to see an example of how they are used, see the An Introduction to Tabular Q-Learning Algorithm as an Adaptive Controller section of the tutorial notebook.

Some things to note when using our current implementation of discretizing spaces:

• It has only been tested against a single-value observation and action space with a very small number of bins. Larger observation and action spaces as well as bins can lead to memory issues when initializing the wrappers, building and updating the Q-Table. For your PPO example, this might not been an issue if you implement your own discretization wrapper or if the PPO algorithm handles the Q-Table more efficiently than we have.
• We have not validated our datasets with discretized spaces thus, it is more likely than not that there are bugs in our existing discretization source code. If you choose to use our implementation for discretization and find any bugs please, feel free to make a pull-request that fixes the bugs as we do not plan to maintain neither the Tabular Q-Learning nor discretization source code ourselves at the moment.