search

SciML / Surrogates.jl

Surrogate modeling and optimization for scientific machine learning (SciML)
https://docs.sciml.ai/Surrogates/stable/
Other
328 stars 70 forks source link

Rewrite MOE algo #394

Closed vikram-s-narayan closed 1 year ago

vikram-s-narayan commented 2 years ago

As mentioned in issue #387, MOE had an issue with accuracy when compared to accuracy of constituent surrogate experts.

This rewrite of the algo significantly improves accuracy. For the example, given in that issue thread earlier here are the results:

Old MOE RMSE: 3.09 Kriging RMSE: 0.10 RBF RMSE: 0.64

Now MOE RMSE: ~ 0.0069

To do:

  1. Register package (I will do this after this PR is merged)
  2. Add documentation (I will do this after registering package as users will need to be able to use MOE)
vikram-s-narayan commented 2 years ago

SurrogatesMOE Tests are passing.

ranjanan commented 2 years ago

okay, I'm checking if these tests fail on master too. In which case we can open a new issue about that and fix. https://github.com/SciML/Surrogates.jl/pull/396

codecov[bot] commented 1 year ago

Codecov Report

Merging #394 (9c9ef77) into master (8f8128d) will increase coverage by 0.25%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #394      +/-   ##
==========================================
+ Coverage   79.40%   79.66%   +0.25%     
==========================================
  Files          16       16              
  Lines        2331     2567     +236     
==========================================
+ Hits         1851     2045     +194     
- Misses        480      522      +42
Impacted Files Coverage Δ
src/Earth.jl 79.37% <0.00%> (-8.56%) :arrow_down:
src/Lobachevsky.jl 93.38% <0.00%> (-3.86%) :arrow_down:
src/Surrogates.jl 18.18% <0.00%> (-3.56%) :arrow_down:
src/Radials.jl 82.75% <0.00%> (-3.10%) :arrow_down:
src/LinearSurrogate.jl 97.61% <0.00%> (-2.39%) :arrow_down:
src/SecondOrderPolynomialSurrogate.jl 98.14% <0.00%> (-1.86%) :arrow_down:
src/GEK.jl 92.94% <0.00%> (-0.73%) :arrow_down:
src/GEKPLS.jl 94.50% <0.00%> (-0.54%) :arrow_down:
src/Wendland.jl 100.00% <0.00%> (ø)
src/PolynomialChaos.jl 0.00% <0.00%> (ø)
... and 3 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

vikram-s-narayan commented 1 year ago

Have fixed Zygote version in [compat] section of Project.toml to Zygote = "= 0.6.40" as a test in SurrogatesPolyChaos errors with higher Zygote versions (higher than 0.6.43). Have raised an issue with Zygote for this.

ChrisRackauckas commented 1 year ago

Test failure?

This version restriction is pretty nasty and will cause some downstream issues to be aware of (@DhairyaLGandhi), so it would be good if Dhariya could investigate a fix or if we could push the Zygote folks to help figure out what's going on here.