Develop a first useful gas analysis notebook

[saroele]

Let's use this issue to discuss the first gas analyses.

[saroele]

The script Gas.ipynb now only contains a general overview of the measurments and a load duration curve, both in kW. There seem to be only 7 valid gas measurements.

I did not follow the discussion about gas analysis on our last meeting. So feel free to add here. I think what makes sense:

• add plot of a typical week
• check baseload to detect standby gas consumption (waakvlam)
• based on summer consumption (and meta-information about how domestic hot water (DHW) is produced) try to obtain the monthly gas consumption for DHW
• then split the total in DHW and heating ...

[Ryton]

Wanted to get started testing / completing notebook, but:

_dataframes.append(fluksoapi.load_csv(fluksoapi.find_csv(path_tocsv, sensor))) springs an index-error on me: --------------------------------------------------------------------------- IndexError Traceback (most recent call last)

in () 2 sensors = hp.get_sensors_by_type('gas') 3 for sensor in sensors: ----> 4 dataframes.append(fluksoapi.load_csv(fluksoapi.find_csv(path_to_csv, sensor))) ... --> 151 return os.path.join(folder, found[0]) IndexError: list index out of range I traced it to the new "FL03001579" // eg with sensor "3e3e6e333de61d67a40e564baad101b". Seems that find_csv finds no files, thus an empty 'found' variable, causing found[0] to be out of range. Proposal: A) add an " elif len(found) < 1" check in find_csv, with warning // fault message instead of crashing. & B) make sure load_csv() gives no dataframe when empty file-name is given. I added this in an updated fluksoapi file, will (try to) make a pull request

[saroele]

ha, that's the flukso of Geert, he just filled out the houseprint a few hours ago. Tomorrow there will be a first csv file... Good solution I think. If you're working directly in the develop branch, no need for a pull request for this small change, just push your branch.

I will review your changed code when it is pushed. Thanks!

On Thu, Oct 23, 2014 at 8:50 PM, Ryton notifications@github.com wrote:

Wanted to get started testing / completing notebook, but:

_dataframes.append(fluksoapi.load_csv(fluksoapi.find_csv(path_tocsv, sensor)))

springs an index-error on me:

IndexError Traceback (most recent call last) in () 2 sensors = hp.get_sensors_by_type('gas') 3 for sensor in sensors: ----> 4 dataframes.append(fluksoapi.load_csv(fluksoapi.find_csv(path_to_csv, sensor))) ... --> 151 return os.path.join(folder, found[0]) IndexError: list index out of range

I traced it to the new "FL03001579" // eg with sensor "3e3e6e333de61d67a40e564baad101b". Seems that find_csv finds no files, thus an empty 'found' variable, causing found[0] to be out of range.

Proposal: A) add an " elif len(found) < 1" check in find_csv, with warning // fault message instead of crashing. & B) make sure load_csv() gives no dataframe when empty file-name is given. I added this in an updated fluksoapi file, will (try to) make a pull request

— Reply to this email directly or view it on GitHub https://github.com/opengridcc/opengrid/issues/19#issuecomment-60289616.

[Ryton]

"Error: Overlapping indiches on dataframe.join (in benchmark example) It took me a while to find the cause, but got it.

Def loadcsv(csv): contains df.columns = [csv.split('')[1]] But, csv also contains the folder name in the front... and my folder happens to have a "_" in the name too... ["_L_scripts/"] Proposal (C): => fluksoapi Updated to count from the back instead. It works for me with index [-7]

[Ryton]

I uploaded a revision just now, of fluksoapi.py, gas, water benchmark & opengrid.config_exampe_file.

Dont know if it all went well. Please let me know if not, and comment/edit the code whereever needed.

[saroele]

I think the upload worked well, i'll check you code the coming weekend. Thanks!

[Ryton]

I just added another upload with 2 files:

• a script (based on the existing "gas script"), named _DemoWaterGasElek
• and a wunderground script to fetch local weather data.

Might be usefull to continue from there imho (see _"Demogaswaterelek.ipn").

• However, especially the used unit conversions should still be checked thoroughly (now made on an educated guess).
• A more elegant "utility selection method" might be valuable as well.

As for your approach: I see some interesting parallells with the analysis I am making on water usage (pie chart script, will commit an update later this month). The focus is slightly different there, however (clustering of profiles in time domain, and mainly comparing with OWN historic consumption (within one household)

based on summer consumption (and meta-information about how domestic hot water (DHW) is produced) try to obtain the monthly gas consumption for DHW then split the total in DHW and heating ... => Might be detected using load curve over a longer period, (eg a week/month). But its tricky. But, especially with a buffer tank in the loop, it will quickly become infeasible to make this estimation with just 1-2 utility measurements. => Further correcting/wheighing of this DHW-baseload-estimate (made in summer) could be made by combining information about measured cold water intake during other periods (e.g in winter), IF the assumption is valid that the ratio of cold water use /DHW usage is constant. (which seems reasonable to me, there should be a major correlation).

[Ryton]

and a few more, with Piechart v0.7 the largest one, specifically designed for water ana lysis (but can be applied to other utility data too.

Feel free to test & comment!