search

smartsystems4u / MORL

Multi-Objective Reinforcement Learning sandbox
MIT License
10 stars 2 forks source link

References for your multi objective RL algorithms? #1

Closed ghost closed 2 years ago

ghost commented 3 years ago

Hi,

Thanks for your code! Could you explain the references for your code of multi objective RL algorithms for Deep Sea Treasure Environment?

smartsystems4u commented 2 years ago

Hi Cubicgate, The algorithms are standard Deep RL alorithms. Descriptions of which you can find here: https://stable-baselines3.readthedocs.io/en/master/guide/rl.html. The only adaptation is that I try to solve multiple agents with each with a different optimal solution so we can construct a Pareto set from them. It is similar to evolutionary approaches (e.g. NSGA-II) but using distributed deep RL or multi-agent RL.

Is this helpful?

ghost commented 2 years ago

Hi @smartsystems4u,

Could you help me to see how your code construct a Pareto set?

Thanks!

smartsystems4u commented 2 years ago

If you look at: https://github.com/smartsystems4u/MORL/blob/8e5e8e8f55f4aa87ab9a9a33e35e0c3e59902686/a3c_dst.py#L163 You'll see that losses, rewards etc. are lists. The set of agents each has it's own optimization process solving for a different linearized multiple objective. Together they the set of agents (when trained) can be used to generate a pareto set for the given DST problem.