What is the purpose of the DETECT_FLAG field?

[zsmith27]

The DETECT_FLAG is part of NYSDEC's standard EDD. The definition I created in the data dictionary is:

An indication if a parameter was detected. This is a field typically provided by laboratories reporting analytical chemistry results.

• Information about NYSDEC's EDD can be found here: https://dec.ny.gov/environmental-protection/site-cleanup/environmental-data-submission#:~:text=The%20new%20May%202024%20edition%20of%20the%20EDD
• The EDD manual can be found here: https://dec.ny.gov/sites/default/files/2024-05/eddmanualnewv2024.pdf

The definition for this field in the EDD manual is:

detect_flag (required) – Choose “Y” from the drop-down menu for yes, the analyte was detected in the sample, or “N” for no, the analyte was not detected in the sample. Select ‘Y’ if the result is above the detection limit but below the quantitation limit. Select ‘<’ or ‘>’ when reporting ignitability tests for results that are less than or greater than the flashpoint. If detect_flag is ‘N’, the interpreted qualifier must be ‘U’ for “Undetected” or ‘R’ for “Rejected data”. Prior to reporting rejected data, please consult with your NYSDEC PM to approve the use of rejected data in an EDD submittal. See Section 4.5.2.11 for information regarding the submittal of rejected data.

However, this field was not retained in the previous lake database. Therefore, there are instances in the Oracle database where there are non-detects identified as "U" in the RESULT_QUALIFIER and LAB_QUALIFIER columns, but the DETECT_FLAG is "not_applicable". "not_applicable" was used as a catchall to categorize data that did not include a DETECT_FLAG field.

There are also non-standard character strings to this field attempting to provide more detail:

1. Reporting Limit
2. less than
3. Not Detected

These character strings should likely be changed to "N". However, we will need to be cautious about this change if there are values in the RESULT_VALUE column.

[zsmith27]

@AleneOnion

[zsmith27]

I have been thinking about removing non-detects ("U") from the RESULT_QUALIFIER field. It seems odd to include non-detects in the same field as reject (R), trend (T), and accepted (A) data. The latter categorize data into bins to inform data usage, while non-detects are additional information about the reported result. It is possible to have non-detects and be R, T, or A.

I propose we use DETECT_FLAG for identifying non-detecs and the RESULT_QUALIFIER to indicate how we recommend the data be used (e.g., R, T, or A).

[AleneOnion]

That makes perfect sense to me