chanzuckerberg / cellxgene-census

CZ CELLxGENE Discover Census
https://chanzuckerberg.github.io/cellxgene-census/
MIT License
83 stars 20 forks source link

[python] dataloader optimization (picking up 1169) #1224

Closed ivirshup closed 2 months ago

ivirshup commented 3 months ago

Building on top of #1169

This PR removes the new argument method (which was undocumented and had no tests atm) and instead using return_sparse_X to control conversion. Previously return_sparse_X controlled whether densification happened in a wrapping iterator, while now a dense array is created without having to make a CSR intermediate.

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 84.84848% with 10 lines in your changes missing coverage. Please review.

Project coverage is 91.15%. Comparing base (f775282) to head (629af82). Report is 2 commits behind head on main.

Files Patch % Lines
...us/src/cellxgene_census/experimental/ml/pytorch.py 84.84% 10 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1224 +/- ## ========================================== - Coverage 91.19% 91.15% -0.04% ========================================== Files 77 79 +2 Lines 5971 6183 +212 ========================================== + Hits 5445 5636 +191 - Misses 526 547 +21 ``` | [Flag](https://app.codecov.io/gh/chanzuckerberg/cellxgene-census/pull/1224/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chanzuckerberg) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/chanzuckerberg/cellxgene-census/pull/1224/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chanzuckerberg) | `91.15% <84.84%> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chanzuckerberg#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.