198808xc / Pangu-Weather

An official implementation of Pangu-Weather
1.11k stars 204 forks source link

About constant mask #4

Open Comma0103 opened 1 year ago

Comma0103 commented 1 year ago

According to the paper and pseudocode, three constant masks are added into input data, however neither of them tells the details. Where to download the constant masks and how to preprocess them? Thanks.

0rhisia0 commented 1 year ago

Bumping this, I can find the land-sea and soil type masks via Copernicus but there is no evidence of an topography mask. Would be helpful if someone can confirm where to find this, and whether these are just element wise additions to the input surface variables or integrated in a different way?

bkf-huawei commented 1 year ago
  1. See https://github.com/pangeo-data/WeatherBench for the topography mask.
  2. Different ways are minor differences, in the version of our paper the masks are regarded as a learnable parameter by mistakes, e.g., it seems that model adjusts them automatically during training while not influencing the accuracy (maybe will bring some advantages when looking at a simple case).
veronicamorelli commented 1 year ago

You can download the single level variables and the constant masks from this link: https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=form Check the sections Soil and Other.

They are named: land_sea_mask, soil_type and orography (renamed as geopotential). The orography should be the topography you asked. It has been renamed as geopotential, check this link: https://confluence.ecmwf.int/pages/viewpage.action?pageId=228854952