[TASK] Multi-objective Optimization for Different Optimizers

[Jderrill]

---

Issue Description

Is your feature request related to a problem? Please describe. I have several current cases that I am working on that could benefit from a multi-objective optimization option.

Describe the solution you'd like It would be useful to have a multi-objective optimization feature in the optimizers (especially gradient descent and GA) with the ability to minimize 2 variables and produce a pareto-front type visualization to observe the optimal configuration.

Describe alternatives you've considered You could probably fake this with a raven running raven architecture. Outer loop as one objective, inner with the other. But there is significant overhead with this for large problems, and visualization of a pareto front may become difficult.

---

For Change Control Board: Issue Review

This review should occur before any development is performed as a response to this issue.

• [ ] 1. Is it tagged with a type: defect or task?
• [ ] 2. Is it tagged with a priority: critical, normal or minor?
• [ ] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
• [ ] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
• [ ] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

---

For Change Control Board: Issue Closure

This review should occur when the issue is imminently going to be closed.

• [ ] 1. If the issue is a defect, is the defect fixed?
• [ ] 2. If the issue is a defect, is the defect tested for in the regression test system? (If not explain why not.)
• [ ] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
• [ ] 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
• [ ] 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

[PaulTalbot-INL]

Another approach we've used in the past is to have the models ensemble with another "post-run" model that combines the objectives into a single weighted target objective; this yields a single optimal location on the pareto front rather than the front itself.

As for returning the pareto front, this sounds more similar to the limit surface search algorithm than the optimization algorithm. That said, there's more similarities than differences between the Optimizer.RavenSampled base class and the LimitSurfaceSearch sampler.