internetofwater / ca-gages

Creative Commons Zero v1.0 Universal
0 stars 1 forks source link

Other Layers to add #12

Open ksonda opened 3 years ago

ksonda commented 3 years ago
ksonda commented 3 years ago
ksonda commented 3 years ago

updated gage list with

ksonda commented 3 years ago

any movement on this @vlzimmer ?

vlzimmer commented 3 years ago

I don't think the GDE index is hosted online anywhere, but I have a shapefile. I'm asking @tcarlin23 and Keenan Smith about it. Here's the dropbox that I found: https://www.scienceforconservation.org/products/groundwater-dependent-ecosystems-data

Same issue with ACE Biodiversity... no layers hosted online https://wildlife.ca.gov/Data/Analysis/ACE#523731770-species-biodiversity

vlzimmer commented 3 years ago

TNC Gage Gap is hosted here: https://gagegap.codefornature.org/ . The gages AND analysis will be replaced by our gage layer and Ted Grantham's UC Berkeley code. But this could stand in for now.

vlzimmer commented 3 years ago

Our Gages Layer, which I think you already have, will be the new working layer. It is not complete. We are reviewing the TNC gages and some extra ones, and making decisions on what to include and how to include them.

tcarlin23 commented 3 years ago

I don't think the GDE index is hosted online anywhere, but I have a shapefile. I'm asking @tcarlin23 and Keenan Smith about it. Here's the dropbox that I found: https://www.scienceforconservation.org/products/groundwater-dependent-ecosystems-data

Honestly, Val, we may need to ask Kirk about this dataset and whether it is still the best one to call upon. It almost seems like TNC is trying to move away from using it since I don't see it readily viewable on their website anymore (https://www.scienceforconservation.org/products/?type=map) and the original DropBox link no longer seems to work. I know the methodology is on the older side (from 2010), but maybe there is another reason why?

It looks like they are relying more upon the iGDE (Indicators of GDEs) now. That can be found at gis.water.ca.gov/app/NCDatasetViewer/

The methodology and findings were presented in 2018. It's on the SharePoint (i02_NCCAG_shapefile in GDE folder).

atcuthbertson commented 3 years ago

Hi all,

DWR does publish the NCCAG dataset here as map and feature services:

https://gis.water.ca.gov/arcgis/rest/services/Biota/i02_NaturalCommunitiesCommonlyAssociatedWithGroundwater/MapServer https://gis.water.ca.gov/arcgis/rest/services/Biota/i02_NaturalCommunitiesCommonlyAssociatedWithGroundwater/FeatureServer

-Aaron

On Thu, Apr 1, 2021 at 7:35 PM tcarlin23 @.***> wrote:

I don't think the GDE index is hosted online anywhere, but I have a shapefile. I'm asking @tcarlin23 https://github.com/tcarlin23 and Keenan Smith about it. Here's the dropbox that I found: https://www.scienceforconservation.org/products/groundwater-dependent-ecosystems-data

Honestly, Val, we may need to ask Kirk about this dataset and whether it is still the best one to call upon. It almost seems like TNC is trying to move away from using it since I don't see it readily viewable on their website anymore (https://www.scienceforconservation.org/products/?type=map) and the original DropBox link no longer seems to work. I know the methodology is on the older side (from 2010), but maybe there is another reason why?

It looks like they are relying more upon the iGDE (Indicators of GDEs) now. That can be found at gis.water.ca.gov/app/NCDatasetViewer/

The methodology and findings were presented in 2018. It's on the SharePoint (i02_NCCAG_shapefile in GDE folder).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/internetofwater/CA-Gages/issues/12#issuecomment-812284061, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACP3EZF3DQGLKNQ3UYDXLJLTGUUQDANCNFSM4ZPF6SBA .

-- Aaron Cuthbertson

vlzimmer commented 3 years ago

Aaron, Unfortunately, DWR does not published the layer titled "groundwater dependent ecosystems summary", which is the index that we want to use. The raw dataset doesn't work for us because we simply don't have time to develop the algorithm to produce an index ourselves (nor do we want to produce our own index - we'd prefer to point to someone else's published body of work).

Todd - I see a half dozen layers on the page you linked but no index (e.g. nothing immediately useable). The TNC dataset is still there and I was able to access the dropbox (but we can ask Kirk too) -> https://www.scienceforconservation.org/products/groundwater-dependent-ecosystems-data

TNC is publishing the interconnected surface water map now too, but it's only Central Valley and meant to be supporting SGMA. https://icons.codefornature.org/

vlzimmer commented 3 years ago

I guess I can upload the GDE layer to IoW for now? @ksonda does that work or do we need to figure out where to host it?

ksonda commented 3 years ago

I guess I can upload the GDE layer to IoW for now? @ksonda does that work or do we need to figure out where to host it?

If it doesn't have a home it can live here. The linked data server will download it from here, but the reference link can be changed. Especially if we get it a persistent identifiers anyway.

vlzimmer commented 3 years ago

@tcarlin23 the ACE biodiversity layers aren't hosted anywhere either, are they?

tcarlin23 commented 3 years ago

@tcarlin23 the ACE biodiversity layers aren't hosted anywhere either, are they?

I have reached out to a contact in our biogeographic data branch. I will let you know.

vlzimmer commented 3 years ago

Ru Roh, I ran into an issue right away. I'll take that layer out for now. Screenshot of the error (>100MB) image

vlzimmer commented 3 years ago

Hmm, more issues - tried 3 or 4 times, with and without VPN. Am I trying to do too much? The commit worked, but the push_to_origin did not, so there's a discrepancy between the commit record and what's actually on the server. How can I get those files on Github?
image

ksonda commented 3 years ago

Can try zipping? Else we will need an alternative, or just the ca open data portal

Sent from my phone

Kyle Onda Data Architect Internet of Water Nicholas Institute for Environmental Policy Solutions Duke University Box 90355 Durham, NC 27708

(919) 613-8747 @.***


From: Valerie Zimmer @.> Sent: Friday, April 2, 2021 11:04:08 PM To: internetofwater/CA-Gages @.> Cc: Kyle Onda @.>; Mention @.> Subject: Re: [internetofwater/CA-Gages] Other Layers to add (#12)

Hmm, more issues - tried 3 or 4 times, with and without VPN. Am I trying to do too much? The commit worked, but the push_to_origin did not, so there's a discrepancy between the commit record and what's actually on the server. How can I get those files on Github? [image]https://urldefense.com/v3/__https://user-images.githubusercontent.com/26662237/113466182-8d79c780-93ee-11eb-9b65-4477f48e383d.png__;!!OToaGQ!9OKmC9kj2wX1lx8atPmQtq79itUhqMa2Qaf5qHY6MD6KSlQKDEQg7PicLe2OAs3xOg$

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/internetofwater/CA-Gages/issues/12*issuecomment-812800725__;Iw!!OToaGQ!9OKmC9kj2wX1lx8atPmQtq79itUhqMa2Qaf5qHY6MD6KSlQKDEQg7PicLe1QglZ8qA$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AKQHTNWWVFY3WA56YYUZQ53TG2ASRANCNFSM4ZPF6SBA__;!!OToaGQ!9OKmC9kj2wX1lx8atPmQtq79itUhqMa2Qaf5qHY6MD6KSlQKDEQg7PicLe0GIGjX-Q$.

ksonda commented 3 years ago

open to other problem solving baout this if still too big to upload after compression. Another open, but not-too-open option could be CUAHSI Hydroshare

vlzimmer commented 3 years ago

Ok - I haven't had time to deal with this, sorry It does appear that Github is not capable of file-sharing on the scale that we'd like to do, so I'll try and figure out the CUAHSI thing next.

-V

On Wed, Apr 7, 2021 at 1:47 PM Kyle Onda @.***> wrote:

open to other problem solving baout this if still too big to upload after compression. Another open, but not-too-open option could be CUAHSI Hydroshare https://www.hydroshare.org/home/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/internetofwater/CA-Gages/issues/12#issuecomment-815251545, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGLNKXKYCQBZ2LZAHAOGREDTHTAD5ANCNFSM4ZPF6SBA .

vlzimmer commented 3 years ago

But really, we're still choosing layers is the other issue, which is the main reason I don't have time - I'm trying to choose layers. We're just too early in the process to be trading layers on a large scale.

On Tue, Apr 13, 2021 at 7:13 AM Val Zimmer @.***> wrote:

Ok - I haven't had time to deal with this, sorry It does appear that Github is not capable of file-sharing on the scale that we'd like to do, so I'll try and figure out the CUAHSI thing next.

-V

On Wed, Apr 7, 2021 at 1:47 PM Kyle Onda @.***> wrote:

open to other problem solving baout this if still too big to upload after compression. Another open, but not-too-open option could be CUAHSI Hydroshare https://www.hydroshare.org/home/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/internetofwater/CA-Gages/issues/12#issuecomment-815251545, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGLNKXKYCQBZ2LZAHAOGREDTHTAD5ANCNFSM4ZPF6SBA .

vlzimmer commented 3 years ago

Can I just FTP or Google Drive or Dropbox some stuff? The CUASHI site thinks I'm the author of anything I upload and it's a process to add stuff. I just want to drag and drop and not write an abstract for it!

ksonda commented 3 years ago

If that's easiest for you for now that's fine, just make sure there's a distinct URL for each layer. (i.e., for GDrive or Dropbox enable download from external users). I wasn't sure what file sharing you had available to you. With CUAHSI you can also just make one entry for the entire project with a pretty minimal abstract and put all the files in it, like this one I made. Up to you!

vlzimmer commented 3 years ago

Well here's the only one that's done so far, but this one has a weblink too - not all of them do (I think) https://www.hydroshare.org/resource/192b23acb7124824a54112a34560f3b8/

I wasn't sure what file sharing you had available to you None, except Teams/Sharepoint, but we cannot share externally. There are literally no file-sharing options that the state of california is officially sanctioning. Including github

vlzimmer commented 3 years ago

Can you see that layer? I think it's private, but the Public and Discoverable Buttons are inactive, so I'm not sure how to make it visible to anyone else unless you have a username.

ksonda commented 3 years ago

Ok. Can you make the resource e their Public or add me as an authorised user?

Sent from my phone

Kyle Onda Data Architect Internet of Water Nicholas Institute for Environmental Policy Solutions Duke University Box 90355 Durham, NC 27708

(919) 613-8747 @.***


From: Valerie Zimmer @.> Sent: Tuesday, April 13, 2021 10:44:32 AM To: internetofwater/CA-Gages @.> Cc: Kyle Onda @.>; Mention @.> Subject: Re: [internetofwater/CA-Gages] Other Layers to add (#12)

Well here's the only one that's done so far, but this one has a weblink too - not all of them do (I think) https://www.hydroshare.org/resource/192b23acb7124824a54112a34560f3b8/https://urldefense.com/v3/__https://www.hydroshare.org/resource/192b23acb7124824a54112a34560f3b8/__;!!OToaGQ!4oTHTlmHvueLwghgAXMHHIQ3gb_gIOrcYM-LMt3ynz67M3c6escB4tqu96W7g_eaYA$

I wasn't sure what file sharing you had available to you None, except Teams/Sharepoint, but we cannot share externally. There are literally no file-sharing options that the state of california is officially sanctioning. Including github

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/internetofwater/CA-Gages/issues/12*issuecomment-818793294__;Iw!!OToaGQ!4oTHTlmHvueLwghgAXMHHIQ3gb_gIOrcYM-LMt3ynz67M3c6escB4tqu96UibBFizg$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AKQHTNSCRE65YXJMPEZQ773TIRKFBANCNFSM4ZPF6SBA__;!!OToaGQ!4oTHTlmHvueLwghgAXMHHIQ3gb_gIOrcYM-LMt3ynz67M3c6escB4tqu96Wp9RFx9g$.

vlzimmer commented 3 years ago

what's your username?

vlzimmer commented 3 years ago

image Inactive Buttons - Public and Discoverable. Do I need to do something else to make them active?

ksonda commented 3 years ago

Ok I see it. To make it public or discoverable it looks like you need to add a keyword (annoying!)

Sent from my phone

Kyle Onda Data Architect Internet of Water Nicholas Institute for Environmental Policy Solutions Duke University Box 90355 Durham, NC 27708

(919) 613-8747 @.***


From: Valerie Zimmer @.> Sent: Tuesday, April 13, 2021 10:51:59 AM To: internetofwater/CA-Gages @.> Cc: Kyle Onda @.>; Mention @.> Subject: Re: [internetofwater/CA-Gages] Other Layers to add (#12)

[image]https://urldefense.com/v3/__https://user-images.githubusercontent.com/26662237/114573161-fbaf5d00-9c2c-11eb-8cc2-50a58c4c5723.png__;!!OToaGQ!7yiIZ8C-VUiMhgwDt17tbQdtFUOrO8bi-ppsfiChJzxMrXrIG0uzRGTKXO3A8mCaUg$ Inactive Buttons - Public and Discoverable. Do I need to do something else to make them active?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/internetofwater/CA-Gages/issues/12*issuecomment-818798947__;Iw!!OToaGQ!7yiIZ8C-VUiMhgwDt17tbQdtFUOrO8bi-ppsfiChJzxMrXrIG0uzRGTKXO2O_NoQ8w$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AKQHTNXWGMJJLZZYOWQKHXDTIRLA7ANCNFSM4ZPF6SBA__;!!OToaGQ!7yiIZ8C-VUiMhgwDt17tbQdtFUOrO8bi-ppsfiChJzxMrXrIG0uzRGTKXO2A7OUnIw$.

vlzimmer commented 3 years ago

There it is! Ok, I'll just use the keyword "SB19" for now. Unique and obscure, so you'll be able to see everything I upload that way.

vlzimmer commented 3 years ago

I've started saving final layers as "Layer Packages" constructed in ArcGIS Pro. This allows easy upload to our ESRI public map for internal organization and anyone else running ArcGIS Pro can see the layer without having to develop symbology (and tells you the attribute of interest). It may not be readable by other software, however (like, for example, ArcMap... headsmack). Let me know if you can read it.

ksonda commented 3 years ago

It is probably possible for an R-based workflow to read these with some work, but saving a version in GeoPackage (.gpkg), geojson, or .shp alongside any .lpk could work

Sent from my phone

Kyle Onda Data Architect Internet of Water Nicholas Institute for Environmental Policy Solutions Duke University Box 90355 Durham, NC 27708

(919) 613-8747 @.***


From: Valerie Zimmer @.> Sent: Tuesday, April 13, 2021 11:07:48 AM To: internetofwater/CA-Gages @.> Cc: Kyle Onda @.>; Mention @.> Subject: Re: [internetofwater/CA-Gages] Other Layers to add (#12)

I've started saving final layers as "Layer Packages" constructed in ArcGIS Pro. This allows easy upload to our ESRI public map for internal organization and anyone else running ArcGIS Pro can see the layer without having to develop symbology (and tells you the attribute of interest). It may not be readable by other software, however (like, for example, ArcMap... headsmack). Let me know if you can read it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/internetofwater/CA-Gages/issues/12*issuecomment-818811315__;Iw!!OToaGQ!7Pq5SOmFz7Zkn5_N3sF2fWVwzk4lkwy3PY-5M5u_U6S9yy_3ld-9w_PpwV9Ga8M0qw$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AKQHTNSHX527ARUT2D4CGXDTIRM4JANCNFSM4ZPF6SBA__;!!OToaGQ!7Pq5SOmFz7Zkn5_N3sF2fWVwzk4lkwy3PY-5M5u_U6S9yy_3ld-9w_PpwV-KvtglcQ$.

vlzimmer commented 3 years ago

How about a .gdb file? I am unable to export anything other that .gdb, .lyrx, or .lypx files from ArcGIS Pro, unless our IT department has a license for an ESRI extension called Data InterOperability. I am checking.

100% of the .gdb files will be > 100 MB. So we will have to use CUASHI to trade files, and I'll try and just upload when we have a "final" dataset, rather than incrementally.

ksonda commented 3 years ago

gdb is ok

vlzimmer commented 3 years ago

I found a workaround for shp.

vlzimmer commented 3 years ago

Ok, the Ecosystem Management simplified layers are uploaded. These are likely to make it into the final analysis, although nothing is final yet. There will be many other layers, but we need to figure out what they are. The zip files are shapefiles.

ksonda commented 3 years ago

@vlzimmer how is everything going? Do you feel like we need to set up another meeting or are you just focusing deciding on layers still?

vlzimmer commented 3 years ago

Hi Kyle,

Yes, we need another meeting but I'm up to my eyeballs in trying to get everything lined up for the report and main effort... I should have more time later.

-Val

On Mon, May 3, 2021 at 7:16 AM Kyle Onda @.***> wrote:

@vlzimmer https://github.com/vlzimmer how is everything going? Do you feel like we need to set up another meeting or are you just focusing deciding on layers still?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/internetofwater/CA-Gages/issues/12#issuecomment-831288865, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGLNKXKRIRPSU6J4TZV3WTLTL2V5HANCNFSM4ZPF6SBA .

ksonda commented 3 years ago

@vlzimmer Don't want to be annoying, but do want to check in. Anything we can help with?

vlzimmer commented 3 years ago

Hi Kyle - should we set up a chat this week? Do you have time? Friday morning, perhaps? If not, Tue Jun 1 is free. I think I have a better idea of what we are doing now, but i'm not sure how to meet the objectives we discussed in the last meeting.

ksonda commented 3 years ago

OK. Friday is tough, but I can do June 1 at 10am, or 12pm pacific

vlzimmer commented 3 years ago

Lets do June 1 at 10am Pacific. Thanks Kyle

On Mon, May 24, 2021 at 10:59 AM Kyle Onda @.***> wrote:

OK. Friday is tough, but I can do June 1 at 10am, or 12pm pacific

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/internetofwater/CA-Gages/issues/12#issuecomment-847228189, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGLNKXKODQXMTQCWAZDNTKLTPKHWRANCNFSM4ZPF6SBA .