Final Reservoirs whose data needs to be parsed

[vishelar]

The attached document contains information on the current sources of reservoirs available. I have some queries which I have mentioned inside the document.

Reservior sources.docx

[vishelar]

Updating the document: Reservior sources.docx

[dmarulli]

Really awesome work @vishelar @Alan-F

To address your questions

• Question at the top of the document

  • I see your point: in the source data, "Dam ID" is a number, while in the current Carto table "dam_id" seems to refer to the source data's "ID" and "Station ID" attributes. To avoid potential confusion, it looks like you have chosen to instead call it _DamABBR. I think that is a good solution. Let's go with that so as not to contradict the source data. I believe we'll just have to update the occurrences of "dam_id" in js/script.js to reflect this change.

• Question 1

  • Since we can't really consider dams without daily data, let's not consider them :)

• Question 2

  • Have you checked if the 36 unique dams represented in the current Carto table are just a subset of the 95 unique dams represented on the daily_res page? My intuition is that they are. Let's just check that for clarity, but continuing with the 95 dam source strikes me as the way to go here.

• Question 3

  • We should be explicit about which attributes are missing. If the missing attributes aren't any of the necessary ones (Capacity, Dam Name, Dam_ABBR, Date, STORAGE, lat, and lng), we should be fine. If we are missing a substantial amount of the necessary attributes, we are probably thinking about something incorrectly. Let's just check to see if missing necessary attributes is a pervasive problem. We can form a plan of attack if it is. But if it's not, then we should be fine.

Quick final note: we'll also need to be collecting latitude and longitude so we can put the reservoirs on a map--don't see those in the demo data.csv ;)

[vishelar]

1. We will check if the 36 unique reservoirs are included in the list of 95 reservoirs and update once done.
2. For the attributes, I think, we will start by parsing all the attributed mentioned in the above document. We will document if any necessary attribute is missing.
3. Yes, we do need to collect latitude and longitude. We will incorporate that while writing the script in Parsekit. We have identified the page from where the lat. and long. will be parsed.

[vishelar]

The 36 unique reservoirs from carto are part of the 95 reservoirs list represented on the daily_res page. Please find attached excel sheet used to check that. reservoir_list_check.xlsx

[vishelar]

In the final parsers created, data about 78 dams is being parsed. The 36 unique dams from the carto are all included in the list. Information about the list and parsers is uploaded here. https://github.com/California-Data-Collaborative/cadc_reservoir_explorer/tree/master/reservior_explorer_parsers