search

ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
34.18k stars 5.8k forks source link

[Feature] support of complicated action space in QMix algorithm in Rllib. #23634

Open hellohawaii2 opened 2 years ago

hellohawaii2 commented 2 years ago

Search before asking

Description

Currently, QMIX algorithm only support Discrete action space. Could you please add the support of Tuple() spaces?

Use case

I customize a environment using PettingZoo, and the action space is MultiBinary. By referring to https://github.com/ray-project/ray/issues/1519, I replace the MultiBinary of action space with nested Tuple of Discrete. However, it seems that QMIX only supports Discrete action space. I got and error in the function _validate(obs_space, action_space), at the beginning of the init function of class QMixTorchPolicy.

Related issues

No response

Are you willing to submit a PR?

gjoliver commented 2 years ago

can you use a wrapper like https://github.com/ray-project/ray/blob/master/rllib/env/wrappers/recsim.py#L130 to solve this?