Estimate the total storage size required by Zarr files

kartoza / tomorrownow_gap

https://kartoza.github.io/tomorrownow_gap/

0 stars 0 forks source link

Estimate the total storage size required by Zarr files #78

Open danangmassandy opened 1 month ago

danangmassandy commented 1 month ago

GAP Data product that will be in zarr:

CBAM Reanalysis Data
Salient Seasonal Forecast Data (includes historical forecast)
Tomorrow.io Short-term Forecast Data (includes historical forecast)

danangmassandy commented 1 month ago

For CBAM Reanalysis Data:

1 year data = 2.3 GB
10 years data = 23 GB

Total time taken to convert 1 year data in staging: ~50minutes.

mvmaltitz commented 6 days ago

CBAM zarr 2012-2023 total size: 29GB Salient zarr 1 forecast_date of 3months data total size: 891MB

danangmassandy commented 3 hours ago

This is blocked by #139. I need to build a zarr from Tomorrow.io short-term forecast for 1 forecast_date, then I'll be able to estimate the total storage size.