NIEHS / beethoven

BEETHOVEN is: Building an Extensible, rEproducible, Test-driven, Harmonized, Open-source, Versioned, ENsemble model for air quality
https://niehs.github.io/beethoven/
Other
5 stars 0 forks source link

Negative PM2.5 #354

Open sigmafelix opened 3 months ago

sigmafelix commented 3 months ago

This is the right time to determine the way to handle negative values in PM2.5 data (cf. #7). Base learner fitting will not make sense unless the negative value problems are resolved.

According to About AQS Data,

In the past, AQS would replace the sample value with ½ of the Method Detection Limit (MDL) if the sample value was below the MDL. AQS no longer does this and historical substitutions have been removed. AQS does store sample specific MDLs where reported and the reference MDL for each method for use when the sample MDL is not reported. The user can make their own substitutions with this data, if desired.

This statement is vague on the disposition of negative values when hourly data are aggregated (it does not apply to 24-hour continuous measurements, of course). Since MDLs are different by measurement methods and devices, we'd better replace all negative values with fairly low values (1 or 2?) or just remove them.