USEPA / EPATADA

This R package can be used to compile and evaluate Water Quality Portal (WQP) data for samples collected from surface water monitoring sites on streams and lakes. It can be used to create applications that support water quality programs and help states, tribes, and other stakeholders efficiently analyze the data.
https://usepa.github.io/EPATADA/
Creative Commons Zero v1.0 Universal
40 stars 18 forks source link

Missing tribal ATTAINS assessment units #547

Open cristinamullin opened 5 days ago

cristinamullin commented 5 days ago

Describe the bug

TADA_GetATTAINS takes a long time to run on tribal WQP datasets and is returning an empty dataframe even though there are relevant assessment units (see ATTAINS tribes in HMW for examples: https://mywaterway.epa.gov/tribe/REDLAKE and https://mywaterway.epa.gov/tribe/SFNOES). This is an issue with all the example tribal data I've tested so far (not just the point data).

To Reproduce

Code to reproduce the behavior:

# tribal example data
# Data_6Tribes_5y <- TADA_DataRetrieval(
#   organization = c(
#     "REDLAKE_WQX",
#     "SFNOES_WQX",
#     "PUEBLO_POJOAQUE",
#     "FONDULAC_WQX",
#     "PUEBLOOFTESUQUE",
#     "CNENVSER"
#   ),
#   startDate = "2018-01-01",
#   endDate = "2023-01-01"
# )
ATTAINS_tribes <- Data_6Tribes_5y
ATTAINS_tribes_2 <- fetchATTAINS(ATTAINS_tribes)
TADA_ViewATTAINS(ATTAINS_tribes_2)
# Error in if (nrow(ATTAINS_table) == 0) { : argument is of length zero
#   In addition: Warning message:
#     There were 2 warnings in `stopifnot()`.
#   The first warning was:
#     ℹ In argument: `!...`.
#   Caused by warning:
#     ! Unknown or uninitialised column: `assessmentunitidentifier`.
#   ℹ Run dplyr::last_dplyr_warnings() to see the 1 remaining warning. 

#############################################################

# takes a long time and results in no ATTAINS catchment found and returns an empty spatial dataframe
ATTAINS_tribes_3 <- TADA_GetATTAINS(ATTAINS_tribes)

#  Red Lake https://mywaterway.epa.gov/tribe/REDLAKE
REDLAKE_WQX_ATTAINS <- TADA_DataRetrieval(organization = "REDLAKE_WQX")
# takes about 30 minutes and returns an empty dataframe
# "There are no ATTAINS catchments associated with these WQP observations. Returning an empty data frame for `TADA_with_ATTAINS`."
REDLAKE_WQX_ATTAINS_2 <- TADA_GetATTAINS(REDLAKE_WQX_ATTAINS)

#############################################################

# sac and fox https://mywaterway.epa.gov/tribe/SFNOES
SFNOES_WQX_ATTAINS <- TADA_DataRetrieval(organization = "SFNOES_WQX")
# "There are no ATTAINS catchments associated with these WQP observations. Returning an empty data frame for `TADA_with_ATTAINS`."
SFNOES_WQX_ATTAINS_2 <- TADA_GetATTAINS(SFNOES_WQX_ATTAINS)

Expected behavior

Most (not all) ATTAINS tribes use their WQP monitoring locations as their ATTAINS assessment units (points - lat/long). I would expect the WQP monitoring locations (points) to match up with the overlapping ATTAINS assessment units (points). They are often the same exact lat/long. Tribal assessment units that are lines/polys should work as well.

Additional context

This is not related to why the ATTAINS spatial data for tribes is not returning, but may still be a good option to try (faster). There is another ATTAINS service that can return the assessment unit and site crosswalk directly if the entity provided it (this crosswalk is required for tribes). We may be able to use that directly instead of, or in addition to, the spatial join. For example, the function could check for a crosswalk in ATTAINS first and use that if available. It’d be missing any new sites but might be a good start. If not available, THEN the function could run as it does now (using ATTAINS catchments to match sites and AUs).

Reminders for TADA contributors addressing this issue

Bug fixes should include all the following work:

cristinamullin commented 5 days ago

Note from Jesse: Tribal AUs do have catchments. The ATTAINS team is in the process of transitioning from one hydrography to another one, which is causing a problem for three of the tribes, but all the others should be working.

kathryn-willi commented 4 days ago

Will explore this as part of my next ATTAINS-related PR!

cristinamullin commented 4 days ago

Will explore this as part of my next ATTAINS-related PR!

Thanks Katie! The easiest way to find tribes that are participating in ATTAINS is through HMW: https://mywaterway.epa.gov/tribe/CHOCNAT. The drop down on that page includes a list of all the current or past ATTAINS tribes.