AdrienWehrle / SICE

Pre-operational Sentinel-3 snow and ice products (SICE) - Geological Survey of Denmark and Greenland (GEUS)
http://snow.geus.dk/
0 stars 0 forks source link

Folders larger than 10Gb ignored although containing data to process #15

Closed BaptisteVandecrux closed 3 years ago

BaptisteVandecrux commented 3 years ago

If I understand it well, the following lines of S3_proc.sh test if the output of gpt is <10Gb. If not it removes the folder.

  # # Discard out bad folders (defined as size > 10 GB)
  (cd ${dest}/../; du -sm * | awk '$1 > 10000 {print $2}' | xargs rm -fr)
  if [[ ! -d "${dest}" ]]; then continue; fi # if we removed the directory, break out of the loop

In my case the folder do contain date. So why this folder should be ignored?

image

AdrienWehrle commented 3 years ago

What are the SICE options you set to obtain those outputs? Your files are really heavy, I usually never have files over 3Mo for Greenland (masked for clouds, but still!).

BaptisteVandecrux commented 3 years ago

This was due to a change of BandSelect output in SNAP8: masks were saved in the geotiff file as well. In https://github.com/GEUS-SICE/SICE/tree/SNAP8 Subset is used instead. Unfortunately per-pixel geocoding does not work in SNAP8, so it uses tie point geocoding.