codeforboston / home-energy-analysis-tool

https://www.codeforboston.org/projects/
MIT License
8 stars 25 forks source link

Frontend: Upload gas-usage CSV and pass to python #84

Open plocket opened 9 months ago

plocket commented 9 months ago

[We should start with EverSource and National Grid.]

plocket commented 8 months ago

I actually think js app could parse the data because we need more than just ~weather data~ [house fuel data] parsed out. Then it could hand just the ~weather data~ [fuel data] to the py rules engine.

stevebreit commented 8 months ago

For clarification, weather data will be obtained from open-meteo via its API. The only data in the .csv files is natural gas usage data. Homeowners obtain this data by signing on to their natural gas utility and requesting a download of up to 36 months of usage in .csv format. There are two major gas utilities in MA: EverSource and National Grid. Each utility allows account owners to download historical natural gas usage in .csv format. The data in the two company’s files is somewhat different: For each gas billing period, EverSource reports the end date of the period and the number of days in the period. This is the information requested on the Natural Gas worksheet of the HLA spreadsheet. Instead, National Grid reports the start and end dates of each billing period and does NOT report the number of days in the period. Obviously this can be calculated from the start and end dates but care must be taken to avoid off-by-one errors. On our shared Google drive space, I have uploaded a HEAT-gas-usage-csv-examples.zip file with 5 examples of Eversource .csv downloads and 7 examples of National Grid downloads. The .zip also includes the HLA beta 7 .xlsx file based on that input data.

It's my guess is that writing an importer for these .csv files would be a great starting task for a new developer on this project because it can be done without too much dependency on other code. Our goal is to NOT expect users to tell us which utility provided the data. Rather, the importer should figure out from reading the file which utility provided it and correctly convert the data to the data structures expected by the runtime engine.

Please contact @stevebreit if you have questions.