argmin-rs / argmin

Numerical optimization in pure Rust
http://argmin-rs.org
Apache License 2.0
972 stars 76 forks source link

Correct line search to search over gradient #354

Closed DevonMorris closed 1 year ago

DevonMorris commented 1 year ago

Previously, the line search operated directly over parameter space. This caused issues when there were different types for parameters vs gradients e.g. Lie Groups. This commit changes line searches to operate over gradient parameterization and incorporates gradient updates to parameters via ArgminScaledAdd.

I'm not sure sure what level of testing is needed for this but it passes all the unit tests with cargo test and I also ran the 3 line search examples and the steepest descent example

codecov-commenter commented 1 year ago

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (2457232) 93.35% compared to head (a801c6b) 93.35%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #354 +/- ## ======================================= Coverage 93.35% 93.35% ======================================= Files 117 117 Lines 18803 18803 ======================================= Hits 17554 17554 Misses 1249 1249 ``` | [Impacted Files](https://app.codecov.io/gh/argmin-rs/argmin/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argmin-rs) | Coverage Δ | | |---|---|---| | [...gmin/src/solver/gradientdescent/steepestdescent.rs](https://app.codecov.io/gh/argmin-rs/argmin/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argmin-rs#diff-YXJnbWluL3NyYy9zb2x2ZXIvZ3JhZGllbnRkZXNjZW50L3N0ZWVwZXN0ZGVzY2VudC5ycw==) | `95.04% <ø> (ø)` | | | [argmin/src/solver/linesearch/backtracking.rs](https://app.codecov.io/gh/argmin-rs/argmin/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argmin-rs#diff-YXJnbWluL3NyYy9zb2x2ZXIvbGluZXNlYXJjaC9iYWNrdHJhY2tpbmcucnM=) | `99.05% <100.00%> (ø)` | | | [argmin/src/solver/linesearch/hagerzhang.rs](https://app.codecov.io/gh/argmin-rs/argmin/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argmin-rs#diff-YXJnbWluL3NyYy9zb2x2ZXIvbGluZXNlYXJjaC9oYWdlcnpoYW5nLnJz) | `73.89% <100.00%> (ø)` | | | [argmin/src/solver/linesearch/morethuente.rs](https://app.codecov.io/gh/argmin-rs/argmin/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argmin-rs#diff-YXJnbWluL3NyYy9zb2x2ZXIvbGluZXNlYXJjaC9tb3JldGh1ZW50ZS5ycw==) | `86.60% <100.00%> (ø)` | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.