openproblems-bio / openproblems

Formalizing and benchmarking open problems in single-cell genomics
MIT License
314 stars 78 forks source link

Daniel strobl hvg conservation fix #785

Closed danielStrobl closed 1 year ago

danielStrobl commented 1 year ago

Submission type

Testing

Submission guidelines

PR review checklist

This PR will be evaluated on the basis of the following checks:

Closes #744

scottgigante-immunai commented 1 year ago

@danielStrobl unfortunately this is not passing tests. Converting back to draft https://github.com/danielStrobl/SingleCellOpenProblems/actions/runs/4045129806/jobs/6956657087

scottgigante-immunai commented 1 year ago

Tests passing at https://tower.nf/orgs/openproblems-bio/workspaces/openproblems-bio/watch/Nl0pbqDX86QsF

scottgigante-immunai commented 1 year ago

We definitely need to add something to the API describing this, otherwise method developers are not aware that there is a bias in the metric. If they use a different flavour of HVG they are punished (maybe?)

codecov[bot] commented 1 year ago

Codecov Report

Base: 94.95% // Head: 94.92% // Decreases project coverage by -0.03% :warning:

Coverage data is based on head (ca5b25b) compared to base (61be1cf). Patch coverage: 100.00% of modified lines in pull request are covered.

:exclamation: Current head ca5b25b differs from pull request most recent head 39cb171. Consider uploading reports for the commit 39cb171 to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #785 +/- ## ========================================== - Coverage 94.95% 94.92% -0.03% ========================================== Files 157 157 Lines 4298 4273 -25 Branches 228 226 -2 ========================================== - Hits 4081 4056 -25 Misses 142 142 Partials 75 75 ``` | Flag | Coverage Δ | | |---|---|---| | unittests | `94.92% <100.00%> (-0.03%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/openproblems-bio/openproblems/pull/785?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio) | Coverage Δ | | |---|---|---| | [...ch\_integration\_feature/metrics/hvg\_conservation.py](https://codecov.io/gh/openproblems-bio/openproblems/pull/785?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#diff-b3BlbnByb2JsZW1zL3Rhc2tzL19iYXRjaF9pbnRlZ3JhdGlvbi9iYXRjaF9pbnRlZ3JhdGlvbl9mZWF0dXJlL21ldHJpY3MvaHZnX2NvbnNlcnZhdGlvbi5weQ==) | `100.00% <100.00%> (ø)` | | | [openproblems/tasks/denoising/methods/alra.py](https://codecov.io/gh/openproblems-bio/openproblems/pull/785?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#diff-b3BlbnByb2JsZW1zL3Rhc2tzL2Rlbm9pc2luZy9tZXRob2RzL2FscmEucHk=) | `100.00% <0.00%> (ø)` | | | [openproblems/tasks/denoising/methods/magic.py](https://codecov.io/gh/openproblems-bio/openproblems/pull/785?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#diff-b3BlbnByb2JsZW1zL3Rhc2tzL2Rlbm9pc2luZy9tZXRob2RzL21hZ2ljLnB5) | `100.00% <0.00%> (ø)` | | | [.../\_cell\_cell\_communication/\_common/methods/liana.py](https://codecov.io/gh/openproblems-bio/openproblems/pull/785?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio#diff-b3BlbnByb2JsZW1zL3Rhc2tzL19jZWxsX2NlbGxfY29tbXVuaWNhdGlvbi9fY29tbW9uL21ldGhvZHMvbGlhbmEucHk=) | `100.00% <0.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openproblems-bio)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

scottgigante-immunai commented 1 year ago

@LuckyMD pending your re-review.

LuckyMD commented 1 year ago

This is just the metric still. Wasn't the PR meant to include the api.py changes as well?

LuckyMD commented 1 year ago

I don't see how this has changed since my last review...

github-actions[bot] commented 1 year ago

Current build status

danielStrobl commented 1 year ago

@LuckyMD added to api now

danielStrobl commented 1 year ago

this only calculates them again in the metric if it's not present in adata.uns.

scottgigante-immunai commented 1 year ago

Passing tests locally. Gotta get this moving so I'm going to merge.