UR 1: Data Ingestion

[mvmaltitz]

Expectation

As a user, I require access to the following datasets which I will access via the API:

1. Ground Observations (TAHMO stations) - csv
2. Ground Observations (TAHMO stations) - API
3. CBAM Daily Historical Reanalysis (2012 - 2023)
4. CBAM Daily Historical Reanalysis (Bias-Corrected) (2012-2023)
5. Arable weather stations
6. Laser Disdrometers
7. Radiosonde Observations (Windborne)
8. CBAM Short-Term weather forecast (14-days)
9. Salient Seasonal weather forecast (3-months)
10. ECMWF ERA5 Land
11. CABI PRISE csv
12. CABI PRISE API

Acceptance Criteria

I can view and access the following datasets within the GAP

• [ ] Ground Observations (TAHMO stations) - csv
• [ ] Ground Observations (TAHMO stations) - API
• [ ] CBAM Daily Historical Reanalysis (2012 - 2023)
• [ ] CBAM Daily Historical Reanalysis (Bias-Corrected) (2012-2023)
• [ ] Arable weather stations
• [ ] Laser Disdrometers
• [ ] Radiosonde Observations (Windborne)
• [ ] CBAM Short-Term weather forecast (14-days)
• [ ] Salient Seasonal weather forecast (3-months)
• [ ] ECMWF ERA5 Land
• [ ] CABI PRISE csv
• [ ] CABI PRISE API

Test Criteria

• [ ] The datasets below are ingested in the GAP
• [ ] Ensure the datasets are stored in the correct format
• [ ] Ensure that the correct attributes are stored
• [ ] Ensure datasets are updated as per "Updated" below

Data Sources

The following datasets need to be ingested into the GAP platform:

Ground Observations (TAHMO stations) - csv

Attribute	Details
Source:	TAHMO Gap Filled Data (csv)
Version:	2018-2024
Resolution:	300+ stations across East Africa
Attributes:	Attribute table
Format:	Database in the GAP
Updated:	Once off ingestion

Ground Observations (TAHMO stations) - API

Attribute	Details
Source:	TAHMO API
Version:
Resolution:	300+ stations across East Africa
Attributes:	Attribute table
Format:
Updated:

CBAM Daily Historical Reanalysis (2012 - 2023)

Attribute	Details
Source:
Version:	2012-2023
Resolution:	4km²
Attributes:	Attribute table
Format:	NetCDF
Updated:	Once off ingestion

CBAM Daily Historical Reanalysis (Bias-Corrected) (2012-2023)

Attribute	Details
Source:	Bias-corrected Reanalysis
Version:	2012-2023
Resolution:	4km²
Attributes:	Attribute table
Format:	NetCDF
Updated:	Once off ingestion

Arable weather stations

Attribute	Details
Source:	API
Version:
Resolution:
Attributes:	Attribute table
Format:
Updated:

Laser Disdrometers

Attribute	Details
Source:	API
Version:
Resolution:
Attributes:	Attribute table
Format:
Updated:

Radiosonde Observations (Windborne)

Attribute	Details
Source:	Windborne API
Version:
Resolution:
Attributes:	Attribute table
Format:
Updated:

• [ ] x, y, z location is updated as the balloon is always moving

CBAM Short-Term weather forecast (14-days)

Attribute	Details
Source:	Tomorrow.io API
Version:
Resolution:	4km²
Attributes:	Attribute table
Format:	ZARR
Updated:	Daily

• [ ] Today attributes are stored in the GAP for historical data

Salient Seasonal weather forecast (3-months)

Attribute	Details
Source:	Salient API
Version:	v9
Resolution:	9km²
Attributes:	Attribute table
Format:	ZARR
Updated:

ECMWF ERA5 Land

Attribute	Details
Source:
Version:
Resolution:
Attributes:
Format:
Updated:

CABI PRISE csv

Attribute	Details
Source:	csv
Version:	sent every 2 weeks from CABI
Resolution:
Attributes:
Format:
Updated:

CABI PRISE API

Attribute	Details
Source:
Version:
Resolution:
Attributes:
Format:
Updated:

[mvmaltitz]

Test:

• [ ] Ground Observations (TAHMO stations) - csv

• [ ] Ground Observations (TAHMO stations) - API

• [ ] CBAM Daily Historical Reanalysis (2012 - 2023)

  77

• [ ] CBAM Daily Historical Reanalysis (Bias-Corrected) (2012-2023)

• [ ] Arable weather stations

• [ ] Laser Disdrometers

• [ ] Radiosonde Observations (Windborne)

• [ ] CBAM Short-Term weather forecast (14-days)

  118

  186

• [ ] Salient Seasonal weather forecast (3-months)

• [ ] ECMWF ERA5 Land

  37

• [ ] CABI PRISE csv

  227

• [ ] CABI PRISE API We do not have access to this yet