Closed wilsonbb closed 5 months ago
Check out this pull request on
See visual diffs & provide feedback on Jupyter Notebooks.
Powered by ReviewNB
Before [d0af3ce0] | After [a693f5d0] | Ratio | Benchmark (Parameter) |
---|---|---|---|
33.5±0.4ms | 33.6±0.5ms | 1 | benchmarks.time_batch |
34.7±0.4ms | 33.9±0.2ms | 0.98 | benchmarks.time_prune_sync_workflow |
Click here to view all benchmarks.
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 95.82%. Comparing base (
d0af3ce
) to head (cd026ca
).
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
As was pointed out in https://github.com/lincc-frameworks/tape/issues/380, it is not uncommon for partitions to be empty, which can lead to unintuitive results for users since
head
by default only looks at the first partitions.To make
head(n)
more intuitive for users we can have it search across all partitions stopping only once there aren
rows to return. This differs from Dask behavior where a user must explicitly choose how many partitions to search across.Warning: this removes the
npartitions
argument previously available inEnsembleFrame.head
Solution Description
Here we iterate over each partition similar to how
head()
until we have found enough rows similar to the implementation in LSDBCode Quality
New Feature Checklist