NasjonaltBakkeSegment / safe_to_netcdf

Python scripts for converting specific Copernicus Sentinel products in Standard Archive Format for Europe (SAFE) to NetCDF/CF
GNU General Public License v3.0
6 stars 5 forks source link

Uniformize compression options for all variables #13

Closed ElodieFZ closed 2 years ago

ElodieFZ commented 3 years ago

Issue #12

Compression options are now the same for all variables: zlib=True, complevel=compression_level For compression level usually used is 7.

The chunking is removed for all variables except the S2 L2A specific layers. Somehow the chunking makes the output file bigger.

For example for the S2 L1C 'reference' product:

ElodieFZ commented 3 years ago

@hevgyrt I would rather have your review before merging has maybe the chunking was set for a very specific purpose?

Also, if we want to run faster, lowering the compression level can be an option. It also makes (as far as I understand) the reading of data afterwards faster. I do not know though how to quantify that. As a quick example, with the lowest compression level (1), the sample file grows 20% in size, but the nc creation takes 25% less time.