search

DOI-USGS / loadflex

Models and Tools for Watershed Flux Estimates
http://dx.doi.org/10.1890/ES14-00517.1
Other
14 stars 17 forks source link

using loadcomp with highly skewed observations; suggest absolute residuals #88

Open miguelcleon opened 8 years ago

miguelcleon commented 8 years ago

Hi @aappling-usgs I'm working with Bill to compute some load estimates and we noticed that for some solutes we have very low values much of the time. Below the detection limit so we are using 1/2 the detection limit (2.5 ug/L), for example with NH4 we report this 192 out of 313 samples. occasionally you jump from 2.5 ug/L to >100 ug/L in the next sample. When using the relative residuals this seems to cause our load values to spike to unreasonable values. I was also getting out of memory errors when attempting to aggregate the predictions. Using the absolute residual option seemed to alleviate this problem. I found this same issue for several other solutes.

Bottom line; Maybe we should suggest using absolute residuals for observations with highly skewed distributions?

aappling-usgs commented 8 years ago

Thanks for the information! Do you have a small example with data that you'd be able to share, either here or offline? I'd love to dig in and see whether the relative residuals method can be fixed.

One thing LOADEST is specially designed to do is handle censored values (those below detection limits). Rather than replacing your censored values with 1/2 the detection limit, consider declaring them censored in the language of rloadest and modeling them that way. See https://github.com/USGS-R/rloadest/blob/master/inst/doc/app3.pdf for an example.

miguelcleon commented 8 years ago

Great, thanks, yes I can share it, I think I'll send you the data offline. Thanks for point out rloadests handling of censored values. I didn't even realize they were censored until I shared the distributions with Bill and he told me all those low values were censored.