feat: support online eval during score

[senwu]

Description of the problems or issues

Is your pull request related to a problem? Please describe. A clear and concise description of what the problem is.

Does your pull request fix any issue. Cite the issue id.

Description of the proposed changes

A clear and concise description of what you propose.

Test plan

A clear and concise description of how you test the new changes.

Checklist

• [ ] I have updated the documentation accordingly.
• [ ] I have added tests to cover my changes.
• [ ] All new and existing tests passed.
• [ ] I have updated the CHANGELOG.rst accordingly.

[codecov[bot]]

Codecov Report

Merging #122 (47486c1) into main (a61b333) will decrease coverage by 3.15%. Report is 1 commits behind head on main. The diff coverage is 44.13%.

:exclamation: Current head 47486c1 differs from pull request most recent head 3dc854d. Consider uploading reports for the commit 3dc854d to get more accurate results

Additional details and impacted files

[](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) ```diff @@ Coverage Diff @@ ## main #122 +/- ## ========================================== - Coverage 92.14% 88.99% -3.15% ========================================== Files 40 40 Lines 2024 2145 +121 Branches 456 503 +47 ========================================== + Hits 1865 1909 +44 - Misses 94 169 +75 - Partials 65 67 +2 ``` | Flag | Coverage Δ | | |---|---|---| | unittests | `88.99% <44.13%> (-3.15%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files Changed](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [src/emmental/model.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21vZGVsLnB5) | `76.54% <22.85%> (-17.25%)` | :arrow_down: | | [src/emmental/learner.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL2xlYXJuZXIucHk=) | `78.75% <100.00%> (+0.42%)` | :arrow_up: | | [src/emmental/meta.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldGEucHk=) | `84.46% <100.00%> (ø)` | | | [src/emmental/metrics/accuracy.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldHJpY3MvYWNjdXJhY3kucHk=) | `100.00% <100.00%> (ø)` | | | [src/emmental/metrics/accuracy\_f1.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldHJpY3MvYWNjdXJhY3lfZjEucHk=) | `100.00% <100.00%> (ø)` | | | [src/emmental/metrics/fbeta.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldHJpY3MvZmJldGEucHk=) | `100.00% <100.00%> (ø)` | | | [src/emmental/metrics/matthews\_correlation.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldHJpY3MvbWF0dGhld3NfY29ycmVsYXRpb24ucHk=) | `100.00% <100.00%> (ø)` | | | [src/emmental/metrics/mean\_squared\_error.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldHJpY3MvbWVhbl9zcXVhcmVkX2Vycm9yLnB5) | `100.00% <100.00%> (ø)` | | | [src/emmental/metrics/pearson\_correlation.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldHJpY3MvcGVhcnNvbl9jb3JyZWxhdGlvbi5weQ==) | `100.00% <100.00%> (ø)` | | | [src/emmental/metrics/pearson\_spearman.py](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2VtbWVudGFsL21ldHJpY3MvcGVhcnNvbl9zcGVhcm1hbi5weQ==) | `100.00% <100.00%> (ø)` | | | ... and [6 more](https://app.codecov.io/gh/senwu/emmental/pull/122?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | |