Updated moe to work with different models

nasa / progpy

The NASA Prognostic Python Packages is a Python framework focused on defining and building models and algorit for prognostics (computation of remaining useful life) of engineering systems, and provides a set of models and algorithms for select components developed within this framework, suitable for use in prognostic applications.

https://nasa.github.io/progpy/

Other

52 stars 7 forks source link

Updated moe to work with different models #110

Closed teubert closed 10 months ago

teubert commented 10 months ago

Previously MoE only worked with homogenous models. Now if a output, etc. is missing it will keep trying the next best model until all are completed.

github-actions[bot] commented 10 months ago

Thank you for opening this PR. Each PR into dev requires a code review. For the code review, look at the following:

[ ] Reviewer (someone other than author) should look for bugs, efficiency, readability, testing, and coverage in examples (if relevant).
[ ] Ensure that each PR adding a new feature should include a test verifying that feature.
[ ] All errors from static analysis must be resolved.
[ ] Review the test coverage reports (if there is a change) - will be added as comment on PR if there is a change
[ ] Review the software benchmarking results (if there is a change) - will be added as comment on PR
[ ] Any added dependencies are included in requirements.txt, setup.py, and dev_guide.rst (this document)
[ ] All warnings from static analysis must be reviewed and resolved - if deemed appropriate.

codecov-commenter commented 10 months ago

Codecov Report

Merging #110 (0ac867c) into dev (70ab878) will not change coverage. Report is 5 commits behind head on dev. The diff coverage is 0.00%.

:exclamation: Current head 0ac867c differs from pull request most recent head fb3f5d2. Consider uploading reports for the commit fb3f5d2 to get more accurate results

@@           Coverage Diff           @@
##              dev     #110   +/-   ##
=======================================
  Coverage   83.54%   83.54%           
=======================================
  Files         100      100           
  Lines       10281    10281           
=======================================
  Hits         8589     8589           
  Misses       1692     1692

Files	Coverage Δ
src/progpy/mixture_of_experts.py	`13.54% <0.00%> (ø)`

matteocorbetta commented 10 months ago

Looks good and works as expected. it took me a little to understand what was going on tho. Can the idea behind the heterogeneous comparison with multi-output models be clarified in the test_moe with more comments? Maybe I did not understand it. If models have different output size, the only thing you can do is rank their performance per output, is that what's happening? I think giving a practical example, instead of random numbers (x0+b, x0+c, with b=0.75, etc) would help the user understand what this is for.