openproblems-bio / openproblems

Formalizing and benchmarking open problems in single-cell genomics
MIT License
287 stars 76 forks source link

[Dimensionality reduction] Fix normalization in baselines #760

Closed scottgigante-immunai closed 1 year ago

scottgigante-immunai commented 1 year ago

Partially addresses #756.

scottgigante-immunai commented 1 year ago

Tests passing at https://tower.nf/orgs/openproblems-bio/workspaces/openproblems-bio/watch/1QXSi2cUIVa1tm

codecov[bot] commented 1 year ago

Codecov Report

Base: 94.80% // Head: 94.78% // Decreases project coverage by -0.01% :warning:

Coverage data is based on head (d87b559) compared to base (413aebd). Patch coverage: 91.66% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #760 +/- ## ========================================== - Coverage 94.80% 94.78% -0.02% ========================================== Files 155 155 Lines 4139 4163 +24 Branches 213 215 +2 ========================================== + Hits 3924 3946 +22 Misses 142 142 - Partials 73 75 +2 ``` | Flag | Coverage Δ | | |---|---|---| | unittests | `94.78% <91.66%> (-0.02%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/openproblems-bio/openproblems/pull/760?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio) | Coverage Δ | | |---|---|---| | [...tasks/dimensionality\_reduction/methods/baseline.py](https://codecov.io/gh/openproblems-bio/openproblems/pull/760/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#diff-b3BlbnByb2JsZW1zL3Rhc2tzL2RpbWVuc2lvbmFsaXR5X3JlZHVjdGlvbi9tZXRob2RzL2Jhc2VsaW5lLnB5) | `91.89% <90.47%> (-1.86%)` | :arrow_down: | | [...nality\_reduction/metrics/root\_mean\_square\_error.py](https://codecov.io/gh/openproblems-bio/openproblems/pull/760/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#diff-b3BlbnByb2JsZW1zL3Rhc2tzL2RpbWVuc2lvbmFsaXR5X3JlZHVjdGlvbi9tZXRyaWNzL3Jvb3RfbWVhbl9zcXVhcmVfZXJyb3IucHk=) | `100.00% <100.00%> (ø)` | | | [...l\_cell\_communication/\_common/metrics/odds\_ratio.py](https://codecov.io/gh/openproblems-bio/openproblems/pull/760/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#diff-b3BlbnByb2JsZW1zL3Rhc2tzL19jZWxsX2NlbGxfY29tbXVuaWNhdGlvbi9fY29tbW9uL21ldHJpY3Mvb2Rkc19yYXRpby5weQ==) | `92.59% <0.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

LuckyMD commented 1 year ago

So this is just the logCPM baseline, but doesn't address the metric issue in general, right? With the normalization added as a layer, this might need to be changed again I guess.

scottgigante-immunai commented 1 year ago

That's correct. Once the prenormalization is added, we can probably remove all but one of these.