search

LSSTDESC / DC2-production

Configuration, production, validation specifications and tools for the DC2 Data Set.
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

Time performance of Dask with Parquet files. #285

Closed wmwv closed 5 years ago

wmwv commented 5 years ago
  1. [x] Time the performance of data retrieval and aggregation with Dask and Parquet files.
  2. [x] Explore performance of simple vs. partitioned ('hive') Parquet files. ~3. [ ] Explore performance of different partioning schemes~

Notes