Fix Setup config + only convert action space when convert is set to true

[visuallization]

This should enable developer to still use discrete action spaces with a size bigger than 2. I tested this with rllib and it seems to work but I am not sure if it will break anything else? What's your take on it?

I also fixed some issues with the initial setup. There was a trouble with ray lib and I had to fix the numpy version. You can read about it here: https://github.com/ray-project/ray/issues/31293#issuecomment-1379007369

Cheers

[edbeeching]

Thanks, looking good. Do you have an env in Godot that uses Discrete spaces > 2 ? Does this not need some changes on the Godot side as well?

[visuallization]

Hey there,

I created this example which uses discrete spaces > 2. The agent learns and I didn't had to change anything in the godot environment. You can checkout the project here.

https://github.com/visuallization/godot-ml-platformer/blob/main/environments/platformer/PlatformerAgent.gd

[visuallization]

The only issue I do get now while training with no convert in rllib is this warning. Not yet sure where and if I can convert the observations to numpy arrays beforehand

[edbeeching]

In this line:

move_action = action["move"] if action["move"] <= 1 else -1

Even though action["move"] can be one of three values: 0,1 or 2. Is it converted to one of two values, -1 or 1?

[edbeeching]

Otherwise I think the logic is fine, it would just be nice to have an example which uses the three distinct actions.

[visuallization]

In this line:

move_action = action["move"] if action["move"] <= 1 else -1

Even though action["move"] can be one of three values: 0,1 or 2. Is it converted to one of two values, -1 or 1?

Ah no actually it just transforms 2 to -1 and keeps 0 and 1 as they are. But I see that it is kind of hard to reason about for such a simple mapping and could need some rewriting :)

[edbeeching]

Ah yes you are right.

[visuallization]

@edbeeching I was also thinking that with this change we should probably also set convert_action_space to True in the stable_baselines_wrapper because it only seems to work with conversion.

[edbeeching]

Yes I agree for the convert option

[edbeeching]

@visuallization do you want to make the same PR for the godot 4 branch?

[visuallization]

@edbeeching yeah, just wanted to do that. Will create a new pr for this

[visuallization]

@edbeeching I created the PR here: https://github.com/edbeeching/godot_rl_agents/pull/79