The continuous action space of PettingZoo Wrapper API should use `np.float32`

[25349023]

Describe the bug The action space for continuous action in the PettingZoo Wrapper API is currently np.int32, leading to unexpected and out of control agent behavior.

In the code ml-agents/ml-agents-envs/mlagents_envs/envs/unity_pettingzoo_base_env.py, line 133, it reads:

if act_spec.continuous_size > 0:
    c_space = spaces.Box(
        -1, 1, (act_spec.continuous_size,), dtype=np.int32
    )
....

However, for the continuous action space, the action should be real numbers, that is, np.float32, instead of integers. Hence, the correct code should be:

if act_spec.continuous_size > 0:
    c_space = spaces.Box(
        -1, 1, (act_spec.continuous_size,), dtype=np.float32
    )
....

To Reproduce Steps to reproduce the behavior: Any multi-agent environments that agents use continuous action space. Then use the PettingZoo Wrapper Python API, for example, UnityAECEnv.

Console logs / stack traces No.

Screenshots No.

Environment (please complete the following information):

• Unity Version: Unity 2022.30f1
• OS + version: Windows 10 / Ubuntu 20.04
• ML-Agents version: 0.30.0 from PyPI
• Torch version: 1.8.1
• Environment: No

NOTE: We are unable to help reproduce bugs with custom environments. Please attempt to reproduce your issue with one of the example environments, or provide a minimal patch to one of the environments needed to reproduce the issue.

[25349023]

In addition, I found that there might be an another bug in ml-agents/ml-agents-envs/mlagents_envs/envs/unity_pettingzoo_base_env.py (line 168-171):

if action.continuous is not None:
    self._current_action[current_behavior].continuous[
        current_index
    ] = action.continuous[0]
....

This line sets the whole action of the agent to the first element of the given action. Assumes that the number of continuous actions is 3, and the given action is [0.3, 0.9, 0.1], then the final action for current agent will become [0.3, 0.3, 0.3], which is not correct.

I'm not sure if it is just a typo and I could delete the [0] directly, or there might be a missing dimension of the given action? (e.g. [[0.3, 0.9, 0.1]] instead of [0.3, 0.9, 0.1])

In my case, I simply delete the [0] and it works like a charm.

[miguelalonsojr]

Please upgrade to the latest version of ML-Agents available on the develop branch of this repo and try again.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please open a new issue for related bugs.

[kin-7777777]

This issue is still not resolved, even on the latest develop branch.

[AmineAndam04]

Hi, how did you circumvent this issue?

[TTMead]

Hi, how did you circumvent this issue?

To avoid forking the repo and making the change locally, I directly reassigned the dtype of the action spaces for each agent after I created the environment.

for agent in env.possible_agents:
    env.action_space(agent).dtype = np.float32