AdrienWehrle / SICE

Pre-operational Sentinel-3 snow and ice products (SICE) - Geological Survey of Denmark and Greenland (GEUS)
http://snow.geus.dk/
0 stars 0 forks source link

GPT outputs 2-band geotiffs, which makes G_align crash #12

Closed BaptisteVandecrux closed 3 years ago

BaptisteVandecrux commented 3 years ago

Example xml:


  <node id="Read_OLCI">
    <operator>Read</operator>
    <parameters>
      <file>${OLCIsource}/xfdumanifest.xml</file>
      <formatName>Sen3</formatName>
      <!-- https://forum.step.esa.int/t/error-when-selecting-multiple-bands-from-slstr-file/15319 -->
      <!-- Sen3 ensures that SLSTR is read as multisize -->
    </parameters>
  </node>

  <node id="Rad2Refl_OLCI">
    <operator>Rad2Refl</operator>
    <sources><source>Read_OLCI</source></sources>
    <parameters>
      <sensor>OLCI</sensor>
      <copyTiePointGrids>true</copyTiePointGrids>
      <copyFlagBandsAndMasks>true</copyFlagBandsAndMasks>
      <copyNonSpectralBands>true</copyNonSpectralBands>
    </parameters>
  </node>

  <node id="Reproject_OLCI">
    <operator>Reproject</operator>
    <sources><source>Rad2Refl_OLCI</source></sources>
    <parameters>
      <crs>EPSG:3413</crs>
      <resampling>Nearest</resampling>
      <noDataValue>NaN</noDataValue>
      <includeTiePointGrids>true</includeTiePointGrids>
    </parameters>
  </node>

  <node id="BandSelect_OLCI_Oa01">
    <operator>BandSelect</operator>
    <sources><source>Reproject_OLCI</source></sources>
    <parameters><sourceBands>Oa01_reflectance</sourceBands></parameters>
  </node>
  <node id="Write_OLCI_Oa01">
    <operator>Write</operator>
    <sources><sourceProduct>BandSelect_OLCI_Oa01</sourceProduct></sources>
    <parameters>
      <file>${targetFolder}/r_TOA_01_x.tif</file><formatName>GeoTIFF</formatName>
    </parameters>
  </node>

  <node id="BandSelect_OZA">
    <operator>BandSelect</operator>
    <sources><source>Reproject_OLCI</source></sources>
    <parameters><sourceBands>OZA</sourceBands></parameters>
  </node>
  <node id="Write_OZA">
    <operator>Write</operator>
    <sources><sourceProduct>BandSelect_OZA</sourceProduct></sources>
    <parameters><file>${targetFolder}/OZA_x.tif</file><formatName>GeoTIFF</formatName></parameters>
  </node>

Output geotiff: image image

Message from G_align:

image

AdrienWehrle commented 3 years ago

Would it be possible to see the error message or the whole log? Here you only provide a warning. If you could also provide the SICE options you used it would help me to reproduce the situation, thank you in advance!

BaptisteVandecrux commented 3 years ago

Posted on STEP: https://forum.step.esa.int/t/gpt-outputs-geotiff-with-corrupted-bands-after-bandextract/27888

potentially causing #13 and #15

BaptisteVandecrux commented 3 years ago

Hi Adrien,

I found out that the extra bands were actually flags and non-spectral data that were somehow attached as bands to the geotiffs. Since these extra bands are saved with float resolution, the size of the geotiff exploded, causing issue #15.

For the processing of SLSTR, I can turn off the copy of flags and non-spectral data:

  <node id="Rad2Refl_SLSTR">
    <operator>Rad2Refl</operator>
    <sources><source>Read_SLSTR</source></sources>
    <parameters>
      <sensor>SLSTR_500m</sensor>
      <copyTiePointGrids>true</copyTiePointGrids>
      <copyFlagBandsAndMasks>false</copyFlagBandsAndMasks>
      <copyNonSpectralBands>false</copyNonSpectralBands>
    </parameters>
  </node>

then it outputs single-bands SLSTR files.

It is not as easy for the data we need from OLCI scenes: We need some of the non-spectral information, but we want to output them into single-band geotiffs.

I am looking for a workaround. In the meantime, can you confirm that in previous versions of snap and gpt, the scripts outputed single-band geotiffs?

AdrienWehrle commented 3 years ago

Hi Adrien,

I found out that the extra bands were actually flags and non-spectral data that were somehow attached as bands to the geotiffs. Since these extra bands are saved with float resolution, the size of the geotiff exploded, causing issue #15.

For the processing of SLSTR, I can turn off the copy of flags and non-spectral data:

  <node id="Rad2Refl_SLSTR">
    <operator>Rad2Refl</operator>
    <sources><source>Read_SLSTR</source></sources>
    <parameters>
      <sensor>SLSTR_500m</sensor>
      <copyTiePointGrids>true</copyTiePointGrids>
      <copyFlagBandsAndMasks>false</copyFlagBandsAndMasks>
      <copyNonSpectralBands>false</copyNonSpectralBands>
    </parameters>
  </node>

then it outputs single-bands SLSTR files.

It is not as easy for the data we need from OLCI scenes: We need some of the non-spectral information, but we want to output them into single-band geotiffs.

I am looking for a workaround. In the meantime, can you confirm that in previous versions of snap and gpt, the scripts outputed single-band geotiffs?

Hi Baptiste,

very interesting but also kind of weird, I indeed never ever faced a similar issue (always obtained single-band files). Do you use the exact same SNAP and gpt specs as the ones specified in README.org?

BaptisteVandecrux commented 3 years ago

Saved multiple band issue here: https://forum.step.esa.int/t/gpt-outputs-geotiff-with-corrupted-bands-after-bandselect/27888/15 workaround leads to issues with pixel geocoding https://forum.step.esa.int/t/pixels-set-to-nan-when-using-pixel-geocoding/28599/2

BaptisteVandecrux commented 3 years ago

Will be solved when switching to https://github.com/GEUS-SICE/SICE/tree/SNAP8