LucasAlegre / morl-baselines

Multi-Objective Reinforcement Learning algorithms implementations.
https://lucasalegre.github.io/morl-baselines
MIT License
271 stars 44 forks source link

Evaluation of the generated Pareto front #96

Closed xwen24 closed 6 months ago

xwen24 commented 6 months ago

Hello! Thank you for your codes!

As a beginner in MORL, I have two questions:

  1. By referring to Xu et al. (2020), "a desired Pareto set approximation is expected to have high hypervolume metric and low sparsity metric." When I use CAPQL to generate Pareto fronts, in many cases, one generated Pareto front may have the highest hypervolume but its sparsity is not the lowest. I am wondering if there is a comprehensive indicator to evaluate the Pareto front.

  2. I applied to CAPQL to a customized environment with four objectives where three of them are always positive and another is between [-1,1]. Below is a generated Pareto front. You can see some points even form a cluster. Theoretically, it should be something like a quarter circle. I think that might be due to my reward setting. Are there any requirements on the range (e.g., all should be positive) or the function format (e.g., linear function. Currently, three of my objectives are quadratic.) of the objective for CAPQL?

output

Thanks in advance!

ffelten commented 6 months ago

Hi :-)!

  1. Performance indicators allow putting a single scalar value giving the quality of a Pareto front (PF). Of course, when you compact a PF (i.e. a set of points) into a single scalar, you lose some information. That means that all performance indicators have pros and cons, and no single performance indicator is best to use in all cases -- that is why we generally use multiple indicators in papers. So my advice is: if possible, plot your PFs, if not, use various indicators.
  2. I'm not sure I understand your question here, but the clustering might not be so clustered when looking at other objective dimensions (obj 3 and 4?). For CAPQL, I don't remember all the requirements out of the top of my head but I think your rewards should be fine (double check on the paper though).

Cheers,

xwen24 commented 6 months ago

Thanks so much for answering my questions!