usgs-makerspace / makerspace-sandbox

Some initial R code for playing with data processing (maybe some light visualization).
Other
0 stars 5 forks source link

Investigate floating lake points #671

Closed lindsayplatt closed 3 years ago

lindsayplatt commented 3 years ago

There are still some points off the coast of CONUS. Likely caused by calculating the centroids of HUC10s from HUC12s.

Please see two functions that do that here: https://github.com/usgs-makerspace/wbeep-viz/blob/master/wu_pipeline/2_process/src/read_prep_files.R#L1-L14

library(sf)
library(dplyr)

# Download and unzip spatial data from
# https://www.sciencebase.gov/catalog/item/5f7fc9a682cebef40f0f1c3b

geojson_fn <- "HUC12_CONUS_Clip_Centroids_v2.geojson"
huc12_centroids <- read_and_parse_huc12(geojson_fn)
huc10_centroids <- calc_huc10_from_huc12(huc12_centroids)
mhines-usgs commented 3 years ago

@lindsayplatt any chance you could post the files somewhere in Teams or sharepoint? It seems like the access to read them is unavailable even if i log in (unless I'm you or the other person listed in the 'readable by').

lindsayplatt commented 3 years ago

posted in Teams

mhines-usgs commented 3 years ago

I got the GeoJSON file from Lindsay, we confirmed it is the latest version from ScienceBase, and there are still points that seem to disagree with their comment that they have eliminated those points. There are points in the original file that represent hucs that include islands/great lakes shorelines or the great lakes themselves still appearing in the GeoJSON before we begin manipulating it for HUC10 display.

MicrosoftTeams-image (13)

lindsayplatt commented 3 years ago

Given a new centroid file that is HUC10 instead of HUC12:

  1. Switch out fetch steps to pull down the HUC10 file instead. If it isn't a zip file, we will need to skip that first step and edit the second.
  2. Delete this function and subsequent step.
  3. Edit the process step and read/parse function to read in the HUC10 specific geojson.
  4. Rerun the map SVG creation steps: see # Recreate maps at https://github.com/usgs-makerspace/wbeep-viz/blob/master/wu_pipeline/README.md#rebuilding-svgs-only
mhines-usgs commented 3 years ago

@lindsayplatt just confirming (making sure @jenniferRapp knows too) that we don't need the data resummarized, just the geospatial data?

jenniferRapp commented 3 years ago

If the HUC12 points represent the total ag use for each HUC12 and they go to a HUC10 spatial GF then they need to summarize the ag water use in the HUC 10. No?

mhines-usgs commented 3 years ago

It sounds like from how Lindsay described it, she was already calculating in her pipeline the summary from huc12-huc10, so we wouldn't have to alter that end of it at all, I think

lindsayplatt commented 3 years ago

Yep, we were already going from HUC12 to HUC10 for water use data and then joining to HUC10 spatial data. Shouldn't need to reaggregate, just rejoin, but even if we did, that is all handled automatically in the pipeline. Thinking an illustration of pipeline steps might help explain here:

image

jenniferRapp commented 3 years ago

Thank you. I figured the WU data were coincident with the Centroids file in the attribute table.

Jen

Jennifer L. Rapp, Physical Scientist Chief, Decision Support Branch USGS Water Mission Area Richmond, VA 23228 804-615-6868 (mobile and home office) jrapp@usgs.govmailto:jrapp@usgs.gov

From: Lindsay (Carr) Platt notifications@github.com Sent: Monday, October 26, 2020 11:46 AM To: usgs-makerspace/makerspace-sandbox makerspace-sandbox@noreply.github.com Cc: Rapp, Jennifer L jrapp@usgs.gov; Mention mention@noreply.github.com Subject: [EXTERNAL] Re: [usgs-makerspace/makerspace-sandbox] Investigate floating lake points (#671)

This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.

Yep, we were already going from HUC12 to HUC10 for water use data and then joining to HUC10 spatial data. Shouldn't need to rejoin, but even if we did, that is all handled automatically in the pipeline. Thinking an illustration of pipeline steps might help explain here:

[image]https://user-images.githubusercontent.com/13220910/97194616-5b49a700-1778-11eb-9052-c35a2e3d8fa1.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/usgs-makerspace/makerspace-sandbox/issues/671#issuecomment-716635602, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIBB4BMOU7FBKCE4VLDYQQ3SMWKSRANCNFSM4SQ3PMIQ.

mhines-usgs commented 3 years ago

the current status of this is that we don't have the new data just yet. waiting on Kristen to help us fix the issues with the sciencebase item.

jenniferRapp commented 3 years ago

@lindsayplatt @mhines-usgs @mwernimont The WU folks are re-mapping all the data to the new WBD HUCs. They hope to be done this week, but it may be early next week before they send the data to Lindsay. The want to wait to release the mappers until these updates are made and implemented in our workflow.