Closed smcclure17 closed 2 years ago
Update datasets in parallel by partitioning the bulk MultiRegionDataset into individual state shards. Currently, this runs the "Update Combined Datasets" action in ~38 minutes (with refreshing datasets) compared to the previous ~80-100 minutes.
Snapshot built using this PR: https://covid-projections-git-bump-snapshot-3560-2346-covidactnow.vercel.app/internal/compare/?left=3558&locations=0&metric=9&right=3560&sort=2
Update datasets in parallel by partitioning the bulk MultiRegionDataset into individual state shards. Currently, this runs the "Update Combined Datasets" action in ~38 minutes (with refreshing datasets) compared to the previous ~80-100 minutes.
Snapshot built using this PR: https://covid-projections-git-bump-snapshot-3560-2346-covidactnow.vercel.app/internal/compare/?left=3558&locations=0&metric=9&right=3560&sort=2