search

USEPA / AMET

Code base for the U.S. EPA’s Atmospheric Model Evaluation Tool (AMET)
https://www.epa.gov/CMAQ/atmospheric-model-evaluation-tool
21 stars 22 forks source link

Apparent discrepency between code and documentation for fractional bias and fractional error #173

Open pavlovc2 opened 10 months ago

pavlovc2 commented 10 months ago

The documentation states that the numerator and denominator should be summed separately to calculate fractional bias and error. image

However, the code for DomainStats in AQ_Misc_Functions.R appears to calculate these metrics as the sum after the fraction is calculated for each observation/model pair. The code also multiplies the result by a 100 to report as a percentage rather than a fraction.

https://github.com/USEPA/AMET/blob/1.5/R_analysis_code/AQ_Misc_Functions.R#L652

fb <- signif(c(fb,(sum((data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val)/2)))/length(data.df$stat_id))*100,3)
fe <- signif(c(fe,(sum(abs(data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val/2)))/length(data.df$stat_id))*100,3)

Am I correct that this is a discrepancy, and can this be updated to be consistent with the correct approach?

wkappel commented 10 months ago

Nathan,

Thanks for your comment. It turns out the definition in the document is the one that is incorrect. The summing should be taking place outside of the brackets (as is done in the actual AMET R code). So, we will note this as a correction needed to this document.

Let me know if you have any further questions. Thanks!

Wyat

pavlovc2 commented 10 months ago

Got it, thank you for the quick clarification!