search

azavea / noaa-hydro-data

NOAA Phase 2 Hydrological Data Processing
11 stars 3 forks source link

Improve compression of Parquet files and document file sizes #22

Closed lewfish closed 2 years ago

lewfish commented 2 years ago

This builds on #12

lewfish commented 2 years ago

Found that appending to Parquet adds new files and does not modify existing ones which is good for S3.