PollyNET / Pollynet_Processing_Chain

NRT lidar data processing program for multiwavelength polarization Raman lidar network (PollyNET)
https://polly.tropos.de/
GNU General Public License v3.0
20 stars 8 forks source link

Compress the NetCDF file size #90

Closed ZPYin closed 3 years ago

ZPYin commented 3 years ago

Problems

Disk usage for one day outputs from Picasso is too much (~ 600 MB for a single polly). The total space for housing outputs from all polly data could be at the magnitude of ~10TB, which is out of the current capacity of the server. Therefore, some controlling methods need to be made.

Suggestions

  1. Store float numbers instead of double numbers, which can save half the disk space.
  2. Adjust the compressing scheme.