@westonslaughter, take a look at the changes to any of the Niwot stream_chemistry munge kernels (process_1_108 is a good example). This domain flags measurements that are below detection limit by inserting e.g. "<0.02" into the data columns. I noticed Spencer was referring to every instance of "<0.XX" individually, which is not future proof, so now there's a wildcard that represents any numeric value, including a possible decimal point.
There were two other issues in these kernels. One inconsequential but confusing to potential copy/pasters: variable_flags_dirty = 'dirty') wasn't doing anything here.
More importantly, I forgot to insert d <- qc_hdetlim_and_uncert(d, prodname_ms = prodname_ms) into these kernels! So BDL datapoints weren't being replaced by their half-detection-limit values.
Nvm re: qc_hdetlim_and_uncert! Forgot I added this to the munge engines, so it's not needed inside kernels in most cases. Though note it is required in kernels that write their output, rather than returning it. ftgrep .R qc_hdetlim to see where it's used.
@westonslaughter, take a look at the changes to any of the Niwot stream_chemistry munge kernels (process_1_108 is a good example). This domain flags measurements that are below detection limit by inserting e.g. "<0.02" into the data columns. I noticed Spencer was referring to every instance of "<0.XX" individually, which is not future proof, so now there's a wildcard that represents any numeric value, including a possible decimal point.
There were two other issues in these kernels. One inconsequential but confusing to potential copy/pasters:
variable_flags_dirty = 'dirty')wasn't doing anything here.More importantly, I forgot to insertd <- qc_hdetlim_and_uncert(d, prodname_ms = prodname_ms)into these kernels! So BDL datapoints weren't being replaced by their half-detection-limit values.Nvm re:
qc_hdetlim_and_uncert! Forgot I added this to the munge engines, so it's not needed inside kernels in most cases. Though note it is required in kernels that write their output, rather than returning it.ftgrep .R qc_hdetlimto see where it's used.