chanzuckerberg / cellxgene-census

CZ CELLxGENE Discover Census
https://chanzuckerberg.github.io/cellxgene-census/
MIT License
72 stars 18 forks source link

[pytorch] Set shuffle as default in pytorch, use new algorithm #1188

Closed ebezzi closed 4 weeks ago

ebezzi commented 4 weeks ago
  1. Set shuffle=True as the default for ExperimentDataPipe
  2. Use the new scatter-gather algorithm by default
  3. Set the soma_chunk_size, shuffle_chunk_count to a sensible default of (64, 2000)
  4. Modify docs to explain the above changes
  5. Set shuffle=False in the tests (to avoid nondeterminism)
codecov[bot] commented 4 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.11%. Comparing base (5fc72e2) to head (6906f20). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1188 +/- ## ========================================== - Coverage 91.15% 91.11% -0.04% ========================================== Files 77 77 Lines 5922 5922 ========================================== - Hits 5398 5396 -2 - Misses 524 526 +2 ``` | [Flag](https://app.codecov.io/gh/chanzuckerberg/cellxgene-census/pull/1188/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chanzuckerberg) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/chanzuckerberg/cellxgene-census/pull/1188/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chanzuckerberg) | `91.11% <100.00%> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=chanzuckerberg#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.