Quansight / lsst_dashboard

LSST Dashboard https://quansight.github.io/lsst_dashboard/
BSD 3-Clause "New" or "Revised" License
8 stars 3 forks source link

Refactor patitioning, and add partitioning CLI #163

Closed timothydmorton closed 4 years ago

timothydmorton commented 4 years ago

This update creates a new DatasetPartitioner object, which manages the repartitioning of butler-read data into kartothek datasets, as well as writing tables of summary statistics. This also adds a lsst_data_repartition CLI, which does both the repartitioning and the writing of stats summaries. The only changes necessary to the dashboard code are renaming the datasets in the kartothek reads, as the dataset names now match the original butler dataset names.