search

LucasAlegre / morl-baselines

Multi-Objective Reinforcement Learning algorithms implementations.
https://lucasalegre.github.io/morl-baselines
MIT License
271 stars 44 forks source link

Multi Discrete action space #98

Closed Afouene closed 5 months ago

Afouene commented 5 months ago

I have a question concerning the multi-discrete action space .I'm supposed to have an action which is a vector not a scalar but for example in pql.py the action is a scalar (for different functions ). multiplying the values of the vector is not logic it's like working on a discrete space but with more actions.

LucasAlegre commented 5 months ago

Some algorithms are designed for discrete action spaces (the actions are integers) while others are designed for continuous action spaces (the actions are real-valued vectors). I am not sure I understood your question, how is the action space of your problem?

Afouene commented 5 months ago

for example i have " self.action_space = spaces.MultiDiscrete([6,5,5]) " .PQL for example deals with multi-discrete space actions but the way the algorithm works is that you multiply the values of the vector' "self.num_actions =np.prod(self.env.action_space.nvec)'".even in the select_action function it returns a single scalar. whereas I need a vector as my action like for example action=[3 1 1]

LucasAlegre commented 5 months ago

Our implementations do not support MultiDiscrete action spaces, unfortunately.