CLOUD_COVERAGE not found in metadata

jteulade commented 3 weeks ago

For a lot of products whose : s3://sertit-projects-archives/projects/2017/AEAG/mathias_AEAG/AEAG/FTP2/FC_41426_PRIO_SO13005352-3-06_DS_PHR1B_201307221050531_FR1_PX_E000N43_1103_01712.zip

I get this error:

Traceback (most recent call last):
  File "E:\JTeulade\repositories\stac\scripts\add_items.py", line 35, in <module>
    main()
  File "E:\JTeulade\repositories\stac\libs\sertit-utils\sertit\s3.py", line 99, in s3_env_wrapper
    return function(*_args, **_kwargs)
  File "E:\JTeulade\repositories\stac\scripts\add_items.py", line 22, in main
    add_items(valid_path_products, S3_TARGET_PATH, utils.get_ids())
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 294, in add_items
    path_products = create_items(valid_path_products, catalog, s3_target_path)
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 205, in create_items
    create_item(
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 132, in create_item
    prod.stac.create_item()
  File "e:\jteulade\repositories\eoreader\eoreader\products\product.py", line 1424, in stac
    self._stac = StacItem(self)
  File "e:\jteulade\repositories\eoreader\eoreader\stac\stac_item.py", line 65, in __init__
    self.eo = EoExt(prod, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\stac\stac_extensions.py", line 79, in __init__
    self.cloud_cover = prod.get_cloud_cover()
  File "C:\Users\jteulade\AppData\Local\anaconda3\envs\arcgispro-eo\lib\site-packages\methodtools.py", line 72, in __call__
    return self.__call__(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\__init__.py", line 34, in wrapper
    return func(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\dimap_v2_product.py", line 1229, in get_cloud_cover
    raise InvalidProductError("CLOUD_COVERAGE not found in metadata!")
eoreader.exceptions.InvalidProductError: CLOUD_COVERAGE not found in metadata!

We can't create a STAC item. A solution can be to catch the InvalidProductError when we try to get the cloud cover here

remi-braun commented 3 weeks ago

The better solution is to find the CLOUD COVER key in these product metadata and update the function accordingly 🙏

jteulade commented 3 weeks ago

I don't find CLOUD COVER neither.. I have these properties:

<Imaging_Quality_Measurement>
      <QUALITY_TABLES>PHR</QUALITY_TABLES>
      <MEASURE_NAME>Cloud_Cotation (CLD)</MEASURE_NAME>
      <MEASURE_TYPE>AUTOMATIC</MEASURE_TYPE>
      <Quality_Mask>
        <Component>
          <COMPONENT_TITLE>Cloud_Cotation Mask</COMPONENT_TITLE>
          <COMPONENT_CONTENT>GML MASK</COMPONENT_CONTENT>
          <COMPONENT_TYPE>ENCAPSULATED</COMPONENT_TYPE>
          <COMPONENT_PATH href="MASKS/CLD_PHR1B_P_201307221050531_SEN_693384101-001_MSK.GML"/>
        </Component>
      </Quality_Mask>
    </Imaging_Quality_Measurement>

remi-braun commented 2 weeks ago

Ok then in this case set 0.0 as per the default function :)

jteulade commented 2 weeks ago

Same problem for

<class 'eoreader.exceptions.InvalidProductError'>: s3://sertit-rms2/archives/rms/2016/EMSN-028-Loiret/EMSN028-AOI-01-FRANCE/02IMAGES/POST/RAW/20160609_de2/DE2_PSH_L1B_000000_20160609T103920_20160609T103923_DE2_10692_0F40.zip -> SPACEMETRIC:SENSOR_AZIMUTH, INCIDENCE_ANGLE or VIEWING_ANGLE not found in metadata!

Traceback (most recent call last):
  File "E:\JTeulade\repositories\stac\scripts\add_items.py", line 35, in <module>
    main()
  File "E:\JTeulade\repositories\stac\libs\sertit-utils\sertit\s3.py", line 99, in s3_env_wrapper
    return function(*_args, **_kwargs)
  File "E:\JTeulade\repositories\stac\scripts\add_items.py", line 22, in main
    add_items(valid_path_products, S3_TARGET_PATH, utils.get_ids())
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 325, in add_items
    path_products = create_items(valid_path_products, catalog, s3_target_path)
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 230, in create_items
    create_item(
  File "E:\JTeulade\repositories\stac\stac\stac.py", line 135, in create_item
    prod.stac.create_item()
  File "e:\jteulade\repositories\eoreader\eoreader\products\product.py", line 1424, in stac
    self._stac = StacItem(self)
  File "e:\jteulade\repositories\eoreader\eoreader\stac\stac_item.py", line 73, in __init__
    self.view = ViewExt(prod, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\stac\stac_extensions.py", line 345, in __init__
    view_az, off_nadir, incidence_angle = prod.get_mean_viewing_angles()
  File "C:\Users\jteulade\AppData\Local\anaconda3\envs\arcgispro-eo\lib\site-packages\methodtools.py", line 72, in __call__
    return self.__call__(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\__init__.py", line 34, in wrapper
    return func(*args, **kwargs)
  File "e:\jteulade\repositories\eoreader\eoreader\products\optical\dimap_v1_product.py", line 310, in get_mean_viewing_angles
    raise InvalidProductError(
eoreader.exceptions.InvalidProductError: SPACEMETRIC:SENSOR_AZIMUTH, INCIDENCE_ANGLE or VIEWING_ANGLE not found in metadata!

sertit / eoreader

CLOUD_COVERAGE not found in metadata #156