opendatacube / datacube-core

Open Data Cube analyses continental scale Earth Observation data through time
http://www.opendatacube.org
Apache License 2.0
509 stars 177 forks source link

lost date time and measurements after ingesting the data using the template s2amsil1c_albers_10.yaml #1515

Closed giswt closed 10 months ago

giswt commented 10 months ago

I ingest S2 into local disk and can not get the date with ERROR "NoneType' object has no attribute 'tzinfo' s2_l2a_jjj is OK but s2a_l2a_jjj_10_32651 can not work!

Error message lost attributes

yaml: source_type: s2_l2a_jjj output_type: s2a_l2a_jjj_10_32651

description: Sentinel-2A MSI L1C 10 metre, 100km tile, Australian Albers Equal Area projection (EPSG:32651)

location: '/datacube/ingested_data' file_path_template: 'S2A_MSI_L2A/S2_JJJ32651{tileindex[0]}{tileindex[1]}{start_time}.nc' global_attributes: title: AIR JJJ Data Cube Sentinel-2A Level-2a summary: Sentinel-2A Multispectral Imager Precision Ortho-corrected data prepared by ESA. source: S2A MSI Level-2a granules history: This data represents a tile and reprojection of Sentinel-2 MSI granule data. institution: ESA instrument: MSI cdm_data_type: Grid keywords: JJJ keywords_vocabulary: GCMD platform: SENTINEL-2A processing_level: L1 product_version: '1.0.0' product_suite: Sentinel-2A project: S2_JJJ publisher_email: 1@1.com publisher_name: JJJ_AIR publisher_url: www.a.com coverage_content_type: physicalMeasurement license: https://creativecommons.org/licenses/by/4.0/ naming_authority: AIR acknowledgment: Sentinel-2 data is provided by the European Space Agency on behalf of the European Commission via download.

storage: driver: NetCDF CF

crs: EPSG:32651 tile_size: x: 100000.0 y: 100000.0 resolution: x: 10 y: -10 chunking: x: 200 y: 200 time: 1 dimension_order: ['time', 'y', 'x']

fuse_data: copy

measurements:

giswt commented 10 months ago

s2aard.txt s2_l2a_jjj in the table 32651netcdf.txt s2a_l2a_jjj_10_32651 in the table

SpacemanPaul commented 10 months ago

Appears to be a bug in dataset generation from ingestion - the ingested dataset is created with no properties section, including no searchable date field.

However:

1) The ingestion workflow has been informally discouraged for many years; 2) The ingestion workflow has been formally deprecated since release 1.8.14 (released June 2023); 3) The ingestion workflow will be removed in the next major ODC release (1.9.0, scheduled early 2024) 4) All organisations supporting ODC development no longer use the ingestion workflow

Fixing this issue is therefore a very low priority for the core development team, although we would be happy to review and merge an external PR fixing this issue.

We strongly suggest you stop using the ingestion workflow and switch to the recommended indexing approach.

giswt commented 10 months ago

Thanks for the information.@SpacemanPaul

Appears to be a bug in dataset generation from ingestion - the ingested dataset is created with no properties section, including no searchable date field.

However:

  1. The ingestion workflow has been informally discouraged for many years;
  2. The ingestion workflow has been formally deprecated since release 1.8.14 (released June 2023);
  3. The ingestion workflow will be removed in the next major ODC release (1.9.0, scheduled early 2024)
  4. All organisations supporting ODC development no longer use the ingestion workflow

Fixing this issue is therefore a very low priority for the core development team, although we would be happy to review and merge an external PR fixing this issue.

We strongly suggest you stop using the ingestion workflow and switch to the recommended indexing approach.