open-austin / project-ideas

:bulb: A place to collect ideas for Open Austin projects
183 stars 25 forks source link

MyCreekATX #15

Closed MyCreekATX closed 9 years ago

MyCreekATX commented 9 years ago

One sentence description A site that aggregates any and all creek data with the goal of better connecting people to creeks in Austin. (Phase 1 - Barton, Bull, Boggy, Onion, Waller, and Shoal Creeks)

Link (more details) http://www.mycreekatx.com/ https://www.facebook.com/mycreekatx https://twitter.com/MyCreekATX

Data categories: -Social media (images, comments, and video pulled for each creek's hashtag) -Watershed Environmental Indicators Data - http://www.austintexas.gov/content/1361/FAQ/14531 -USGS Flow data (look at the Colorado River Basin) - http://waterdata.usgs.gov/tx/nwis/current/?type=flow&group_key=basin_cd&site_no_name_select=siteno -Raw Cultural/Social Value Data (not yet analyzed and ongoing data gathering) - https://docs.google.com/spreadsheets/d/1MGOipHCvQcpLbjfCBBpuMfm0ZqRdwmF9oxII79dMbDI/edit?usp=sharing Mapping - ftp://ftp.ci.austin.tx.us/GIS-Data/Regional/coa_gis.html

Project Needs (dev/design/resources) hackathon organization and attendance, preliminary development

Status in progress

luqmaan commented 9 years ago

Do you have any links to creek data?

mateoclarke commented 9 years ago

:+1: DATA LINKS

And can you share the list of creeks you wanted to focus on in the first iteration?

MyCreekATX commented 9 years ago

Let me know if there is anything else you guys need!

mateoclarke commented 9 years ago

Here is another dataset I'm aware of for Water Quality. https://data.austintexas.gov/Environmental/Water-Quality-Sampling-Data/5tye-7ray

We'll need more context to understand which of these measures are good back or in between, and how to make this info helpful & accessible to the average person. Maybe there is some metadata explaining this stuff somewhere?

MyCreekATX commented 9 years ago

Hey Mateo, That data set is a small piece of what goes into the data set I posted. The EII includes various factors other than water quality - for instance, they sample benthic organisms for diversity, type, etc. The EII is a combined scoring system for all ecological variables. And it's set on a 100 point scale so we should be able to easily average the creek overall (rather than the reaches they have listed) and format that score in a user-friendly way. I can clarify further if we need to. Here is the 200 page methodology for the EII data - https://drive.google.com/file/d/0B5nTmYK1N7myOUhRWWFWZEpQbEk/view?usp=sharing.

luqmaan commented 9 years ago

Ok so far we have one dataset:

Do you know of any others @MyCreekATX?

MyCreekATX commented 9 years ago

When you say dataset... do you mean raw data? in spreadsheet format? I added some links to data in the description above but not in a raw format.

mateoclarke commented 9 years ago

Which one is the EII dataset? Is this an example of an EII report? http://www.austintexas.gov/sites/default/files/files/Watershed/eii/Barton_EII_ph1_2011.pdf PDF is a pretty useless way to share data for our purposes. Do you think there is a raw data file that we could get access to that they used when creating this PDF?

And are you aware of a method to extract data from the USGS site? It looks like you'd have to build a scrapper to automate the data collection process for this data to be valuable in real time. I could be wrong...

When we say data set, we mean raw data in a machine-readable format. Here are some formats in a loose order of preference:

  1. JSON
  2. XML
  3. RSS
  4. CSV (or another x seperated variable format)
  5. Google Spreadsheets or Excel file
  6. HTML (requires a web scrapper to be programmed)
MyCreekATX commented 9 years ago

I think we can get the raw data from the EII dataset. I can email someone to get it. Someone I was speaking to at Austin Coding Academy seemed to think there was a way to mine the data on the USGS site easily... I definitely could be wrong.

KATIE COYNE MyCreekATX.com 561.339.5712 facebook https://www.facebook.com/mycreekatx . twitter https://twitter.com/MyCreekATX

On Wed, Jul 15, 2015 at 11:05 AM, Mateo Clarke notifications@github.com wrote:

Which one is the EII dataset? Is this an example of an EII report?

http://www.austintexas.gov/sites/default/files/files/Watershed/eii/Barton_EII_ph1_2011.pdf PDF is a pretty useless way to share data for our purposes. Do you think there is a raw data file that we could get access to that they used when creating this PDF?

And are you aware of a method to extract data from the USGS site? It looks like you'd have to build a scrapper to automate the data collection process for this data to be valuable in real time. I could be wrong...

When we say data set, we mean raw data in a machine-readable format. Here are some formats in a loose order of preference:

  1. JSON
  2. XML
  3. RSS
  4. CSV (or another x seperated variable format)
  5. Google Spreadsheets or Excel file
  6. HTML (requires a web scrapper to be programmed)

— Reply to this email directly or view it on GitHub https://github.com/open-austin/project-ideas/issues/15#issuecomment-121662674 .

mateoclarke commented 9 years ago

Hi Katie,

Any update on the raw data from EII? Is that a Watershed Protection dataset?

Also any more research around getting data in a better format from USGS? If this is crucial data and it needs to be scraped, that would be an immediate first need.

You also linked to the City GIS site. What specific geographic data layers did you find to be relevant?

durs125 commented 9 years ago

Have you used any of the TCEQ data? https://www.tceq.texas.gov/waterquality/monitoring/cwqmn-data-links

MyCreekATX commented 9 years ago

We haven't but it's definitely something we can keep in mind. We've been focusing on the data from the Department of Watershed Protection at the city because they have a complete data set for all creeks and it is updated every two years.

luqmaan commented 9 years ago

The MyCreekATX Charrette & Hack will be held Saturday October 24, 8 AM at 816 Congress Ave, 3rd Floor, Austin, TX, US. https://www.facebook.com/events/1482064512089654/

Closing this issue. Feel free to reopen.