search

GFDRR / CCDR-tools

Geoanalytics for climate and disaster risk screening
https://gfdrr.github.io/CCDR-tools/
12 stars 8 forks source link

Accessing and processing Fathom 3 data #30

Open matamadio opened 8 months ago

matamadio commented 8 months ago

Fathom 3 purchase is completed and the data is currently on dropbox. Soon it will be moved to Azure data lake. The data is split into 1 degree tiles for all the world, for each scenario. A csv lists the ISO-a2 country code for each tile.

We need an automatic selector of country, scenarios, and RPS to download the tiles and the get them directly into the processing.

It is unlikely we will be able to consider the full range of scenarios (280 layers) for each analysis; propose a selection.

Type Period Scenario Defence
Fluvial 2020 SSP 1/2.6 Undefended
Pluvial 2030 SSP 2/4.5 Defended
Coastal 2050 SSP 5/8.5

Return periods: 5, 10, 20, 50, 100, 200, 500, 1000

Fluvial: 112 global layers 2020: 2x8 2030: 2x3x8 2050: 2x3x8

Pluvial: 56 global layers Note: there is no pluvial undefended; only defended option. 2020: 1x8 2030: 1x3x8 2050: 1x3x8

Coastal: 112 global layers 2020: 2x8 2030: 2x3x8 2050: 2x3x8

matamadio commented 7 months ago

Data is in transfer. A dedicated analytitical workflow is required. The plan is to stem a modified version of current parallel code whith slightly different options, expecting the data format and structure of the latest data. It could work with tiles/virtual rasters, instead of merging the tiles into a larger file. This might also make the parallel processing easier/faster. It should also include the options for running on multiple future scenarios.

So user options should be:

Current stem with minor edits from the original: https://github.com/GFDRR/CCDR-tools/tree/main/Top-down/FathomV3

@artessen your opinions?