search

JCSDA-internal / ioda-converters

Various converters for getting obs data in and out of IODA
9 stars 5 forks source link

Error specification in airnow converter #1569

Open jeromebarre opened 4 weeks ago

jeromebarre commented 4 weeks ago

ioda_data[(variable, obsErrName)] = np.full(nlocs, 0.1, dtype=np.float32) https://github.com/JCSDA-internal/ioda-converters/blob/2dddebc903ab8308317a52ddc109e810254d40af/src/compo/airnow2ioda_nc.py#L293

Not sure we should put an absolute 0.1 like this as this would vary per pollutant. Relative error (i.e. a fraction of the obs value) are one thing but can lead to generating noise, as it would give more weight to background measurements which could be under the precision capability of the site.

There are papers out there that deal with this. @hongli-wang @ytangnoaa what are you using for error specification in you airnow assimilation with JEDI?

hongli-wang commented 4 weeks ago

The error model I used is: pm_error=base_value+percentage*obs_value. This is similar to the one in GSI. The representativeness error could be considered but not at present.

ytangnoaa commented 4 weeks ago

The value of 0.1 is the minimal observation error. It contain only instrumentation error, which should be < 10% of observed PM2.5 values according to https://www.epa.gov/aqs/aqs-memos-technical-note-reporting-pm25-continuous-monitoring-and-speciation-data-air-quality

Users can replace it with their flavored observation errors

jeromebarre commented 4 weeks ago

Do you know what would be base_value and percentage are for all pollutants reported in airnow? I don't think we should use 0.1 for all species.

hongli-wang commented 4 weeks ago

Here is the GSI default: percentage 1%. The base value read in from configuration file, which is 1.5. There is another parameter to adjust the percentage item, which is 0.75 in the configuration file. The representativeness error could be considered too. My current setup: the percentage is up to 5% with a minimum value. I will check the value later.

jeromebarre commented 4 weeks ago

Table 2 in https://pmc.ncbi.nlm.nih.gov/articles/PMC5054062/ is one source of information for this. It includes O3, NO2 and PM2.5. Error is mostly relative here no CO or SO2.

ytangnoaa commented 4 weeks ago

If the super observation or merged observation is used, I use the corresponding observation standard deviation within that grid cell for the representation error.