Augmented Lagrangian Development Branch

[alphaville]

Work in progress - do not merge yet

• [x] TODO: Once we merge the other PRs into master, we need to merge master into dev/alm.
• [ ] TODO: I deactivated the Python tests for now; we need to re-enabled them later

Rust implementation (basic steps):

• [x] Define problem
• [x] Define ALM/PM cache
• [x] Define AlmOptimizer interface
• [x] Project on Y
• [x] Solve inner problem using PANOC
• [x] Update Lagrange multiplier vector
• [x] Compute quantities for termination criteria
• [x] Termination criteria
• [x] Gradual decrease of tolerance
• [x] Update penalty parameter
• [ ] Unit tests of the components of the algorithm
  • [x] compute_alm_infeasibility
  • [x] compute_pm_infeasibility
  • [x] update_lagrange_multipliers
  • [x] project_on_set_y
  • [ ] solve_inner_problem
  • [x] is_exit_criterion_satisfied
  • [x] is_penalty_stall_criterion
  • [x] update_penalty_parameter
  • [x] update_inner_akkt_tolerance
  • [x] final_cache_update
• [ ] Outer loop
• [ ] Check whether there is available time
• [ ] Numerical unit tests