internetofwater / geoconnex.us

URI registry for https://geoconnex.us based URIs
Other
24 stars 15 forks source link

EPA Assets #53

Open ksonda opened 4 years ago

ksonda commented 4 years ago

This is a summary of some ideas fleshed out with @dyoung440 and @mthawley (who should feel free correct anything or flag any issues).

Timeline is we can probably begin addressing 1 and maybe 2 the week of June 22.

1. "How's My Waterway?" v2.0

This application, when released, will have landing pages about HUC12s that includes information about it's pollution/impairment assessment status and links to pages about relevant ECHO discharge points and public waters systems.

for geoconnex.us:

Question: Not sure how to link to the rest of geoconnex assets exactly since no HUC12 LC, unless you wanted to index to NLDI @dblodgett-usgs ?

Possible Big Issue: HUCs change over time? Need to resolve what version is being used. :/

Also: EPA assesses "waters'" impairment status. These "Waters" are composed of combinations of reachcodes that may cross HUCs, and could have landing pages in How's My Waterway and/or elsewhere. These could have /epa/assessed-waters/ (or similar) PIDs. Unclear what other links could be, or if reasonable to index to NLDI, etc.

2. Permitted Discharge Locations.

Permitted Discharge Locations (points) have landing pages as a small subset of the larger ECHO system. Since States and Tribes are often enforcing CWA regulations at such facilities, there may be other organizational pages about the same locations.

Thus, for geoconnex:

  1. Reference Locations: create /ref/ set of discharge points at info.geoconnex.us that are subjectOf--

  2. Index the reference locations to NLDI?

3. Community Water System intakes.

All community water systems have water intakes. The exact location data for surface intakes/ wells may be a security issue, but EPA does keep a bunch of important data about them, and has these indexed to HUC12 (for How's My Waterway) and county. The intakes themselves may not have unique identifiers. It's more that Each HUC12/ County will have a list of CWS that have intakes within that geography.

For geoconnex:

dblodgett-usgs commented 4 years ago

Re: HU12s

I created this namespace: https://github.com/internetofwater/geoconnex.us/blob/master/namespaces/nhdplusv2/huc12pourpoints.xml which resolves PIDs like: https://geoconnex.us/nhdplusv2/huc12/0701010101 and redirects to the NLDI (for now). Long run, we absolutely need better landing content for these HUs, whether hosted in the NLDI or elsewhere. For now, just getting the links established is good, IMHO.

Re: HUCs change over time

This issue is settling down with a target to not change. However, in the near term, we should focus on the NHDPlusV2 snapshot and look to a future NHDPlusHR integrated WBD for stability.

Re: waters that span multiple hydrologic units

This is tough -- I think the most practical thing near term is going to be listing multiple associations between impaired waters and hydrologic units as appropriate.

Re: Permitted discharges

I love it. Perfect application of that patter.

Re: community water systems

I think the intakes issue is a little harder. Do they have landing pages? I don't think they belong in the /ref/ namespace until we have a little more certainty about the data from a community perspective. If they have landing pages -- I'd say create an EPA PID for them and get them linked up to the HMW pages.

ksonda commented 4 years ago

Re: community water systems

Intakes don't have landing pages although the CWS do at EPA SDWIS (although among their required query parameters is population) https://enviro.epa.gov/enviro/sdw_report_v3.first_table?pws_id=NM3510701&state=NM&source=Surface%20water&population=601983&sys_num=0

States also have often have separate pages: https://dww.water.net.env.nm.gov/DWW/JSP/WaterSystemDetail.jsp?tinwsys_is_number=35&tinwsys_st_code=NM&wsnumber=NM3510701%20%20%20&DWWState=NM

Also starting some work on API Management platforms with ECOS that might end-run SDWIS and be able to produce selfie-complian LC out of State SDWIS database

ksonda commented 4 years ago

Re: HU12s

I created this namespace: https://github.com/internetofwater/geoconnex.us/blob/master/namespaces/nhdplusv2/huc12pourpoints.xml which resolves PIDs like: https://geoconnex.us/nhdplusv2/huc12/0701010101 and redirects to the NLDI (for now). Long run, we absolutely need better landing content for these HUs, whether hosted in the NLDI or elsewhere. For now, just getting the links established is good, IMHO.

Do you mean just minting PIDs for HMW for now or getting those pages indexed to NLDI somehow?

dblodgett-usgs commented 4 years ago

I meant the HMW could link to those HU12 identifiers.

ksonda commented 3 years ago

This is the NPDES permit file (updated weekly). We should see if there's landing pages in ECHO that are easy to map https://echo.epa.gov/tools/data-downloads/icis-npdes-discharge-points-download-summary. This would also hugely valuable to pair with WQP in NLDI.

jkreft-usgs commented 3 years ago

NPDES was in a test version of NLDI previously. There was a geojson endpoint somewhere, let me see if I can dig it up.

ksonda commented 3 years ago

in any case it should probably be in /epa namespace, unless we can justify /ref/npdes. @dyoung440 may want to weigh in

jkreft-usgs commented 3 years ago

digging through old emails, it was was an s3 bucket and long gone. There are absolutely mappable echo urls though.

ksonda commented 3 years ago

digging through old emails, it was was an s3 bucket and long gone. There are absolutely mappable echo urls though.

Ah well. If the mapping is easy we can put it together. One other consideration is the point type, should we do all of them or only the actual Discharge outfalls? Maybe need to pull in Dwane via email.

ksonda commented 3 years ago

It looks like the ECHO pages are for facility (eg, a WWTP, or a factory) and not for the actual permitted feature (i.e. outfall). hmmmm. THere is a m:1 linkage between outfalls and facilities. I think I'll summarise in an email for Dwane. One option for the short term/ experimental basis would be for IoW to host dummy landing pages for outfalls with links to the corresponding facility landing pages and/or Envirofacts API calls, similar to what we did for WaDE (https://wade-test.geoconnex.us)

dblodgett-usgs commented 3 years ago

I found the same emails @jkreft-usgs -- The network locations came from the EPA Reach Address Database. https://www.epa.gov/waterdata/reach-address-database-rad

That is probably something we should be pulling in as possible.

ksonda commented 3 years ago

welp been 6 months. Seems like EPA WATERS has its own internal NLDI anyway?

https://watersgeo.epa.gov/openapi/waters/#/Discovery/x879

But we could get the NPDES points along with their COMIDs and measures from this one

https://watersgeo.epa.gov/openapi/waters/#/Utility/x613

ksonda commented 3 years ago

Got it https://ofmext.epa.gov/FLA/www3/state_files/national_single.zip

That has a field "REGISTRY_ID" which can be piped into https://echo.epa.gov/detailed-facility-report?fid={REGISTRY_ID}

Also has a field "PGM_SYS_ACRNMS" which is you trim the "NPDES:" from, yields an ID that can also be piped into the same ECHO pattern, as well as the watersgeo API to get the lat/lon, COMIDs and measures

ksonda commented 2 years ago

I want to reorganize this, I think it would be pretty easy.

EPA has evidently already cataloged everythign that is in their WATERS Viewer

image

Just want to verify how persistent Source_FeatureID is. If it's persistent then can do regex. If it's not, and need to use Permanent_Identifier or Feature_Permanent_Identifier, then they probably need 1:1 ids.