search

facebook / Ax

Adaptive Experimentation Platform
https://ax.dev
MIT License
2.38k stars 312 forks source link

per-metric model selection in MBM #3032

Closed sdaulton closed 1 week ago

sdaulton commented 3 weeks ago

Summary: This diff adds support for per-metric model selection in MBM using leave-one-out cross-validation. This will automatically select the best model for each outcome (among specified choices for each outcome) according to a specified criterion (e.g. rank correlation) using leave-one-out cross-validation. The selected models are then assembled into a ModelListGP.

Differential Revision: D64971370

facebook-github-bot commented 3 weeks ago

This pull request was exported from Phabricator. Differential Revision: D64971370

codecov-commenter commented 3 weeks ago

Codecov Report

Attention: Patch coverage is 98.26087% with 4 lines in your changes missing coverage. Please review.

Project coverage is 95.72%. Comparing base (7b671dc) to head (f084a07).

Files with missing lines Patch % Lines
ax/models/torch/botorch_modular/surrogate.py 94.59% 4 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #3032 +/- ## ========================================== + Coverage 95.69% 95.72% +0.02% ========================================== Files 488 488 Lines 49510 49680 +170 ========================================== + Hits 47381 47557 +176 + Misses 2129 2123 -6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

facebook-github-bot commented 3 weeks ago

This pull request was exported from Phabricator. Differential Revision: D64971370

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D64971370

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D64971370

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D64971370

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D64971370

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D64971370

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D64971370

facebook-github-bot commented 1 week ago

This pull request has been merged in facebook/Ax@a0f3b45ca930fd3f4a2af7364b21332f5040f25e.