search

impactlab / thermostat

:bangbang: MOVED TO https://github.com/openeemeter/thermostat - Python package for calculating connected thermostat temperature/run-time field savings metrics
4 stars 1 forks source link

Daylight savings time causes error on import #4

Open mblasnik opened 9 years ago

mblasnik commented 9 years ago

Here is what I typed into python, taken from the sample cod. , just changed file name to my data file names but I'm wondering if there is anything else I should change. I can send you the two csv files if that would help.

import sys import os from os.path import expanduser from thermostat.importers import from_csv from thermostat.metrics import calculate_epa_draft_rccs_field_savings_metrics from thermostat.metrics import seasonal_metrics_to_csv

data_dir = os.path.join(expanduser("~"),"Downloads/") thermostats = from_csv(os.path.join(data_dir,"tstat_meta_sample6.csv"), os.path.join(data_dir,"tstat_interval_sample6.csv"))

for thermostat in thermostats: metrics = calculate_epa_draft_rccs_field_savings_metrics(thermostat) filepath = os.path.join(datadir,"thermostat{}_output.csv".format(thermostat.thermostat_id)) seasonal_metrics_to_csv(metrics,filepath)

Here is the error I'm getting:

Traceback (most recent call last): File "", line 2, in File "/usr/local/nest-python-local/lib/python/site-packages/thermostat/metrics.py", line 33, in calculate_epa_draft_rccs_field_savings_metrics for cooling_season, season_name in thermostat.get_cooling_seasons(): File "/usr/local/nest-python-local/lib/python/site-packages/thermostat/core.py", line 349, in get_cooling_seasons total_cooling = pd.DataFrame(cooling_columns).sum(axis=0) File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 243, in init arrays, columns = _to_arrays(data, columns, dtype=dtype) File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 4748, in _to_arrays dtype=dtype) File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 4841, in _list_of_series_to_arrays indexer = indexer_cache[id(index)] = index.get_indexer(columns) File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/index.py", line 1284, in get_indexer raise InvalidIndexError('Reindexing only valid with uniquely' pandas.core.index.InvalidIndexError: Reindexing only valid with uniquely valued Index objects

philngo commented 9 years ago

Happy to take a look if you send over the files. At first glance, it looks like there is a duplicate timestamp somewhere.

mblasnik commented 9 years ago

Aha, I don't know how they got in there, but there are a handful of duplicate records. I'll fix it up and give it another try.

Thanks,

Michael

On Thu, Jul 30, 2015 at 11:00 AM, Phil Ngo notifications@github.com wrote:

Happy to take a look if you send over the files. At first glance, it looks like there is a duplicate timestamp somewhere.

— Reply to this email directly or view it on GitHub https://github.com/impactlab/thermostat/issues/4#issuecomment-126360191.

mblasnik commented 9 years ago

Tracked it down -- it was the 25 hour day for the end of daylight savings time that messed things up.

On Thu, Jul 30, 2015 at 11:04 AM, Michael Blasnik mblasnik@nestlabs.com wrote:

Aha, I don't know how they got in there, but there are a handful of duplicate records. I'll fix it up and give it another try.

Thanks,

Michael

On Thu, Jul 30, 2015 at 11:00 AM, Phil Ngo notifications@github.com wrote:

Happy to take a look if you send over the files. At first glance, it looks like there is a duplicate timestamp somewhere.

— Reply to this email directly or view it on GitHub https://github.com/impactlab/thermostat/issues/4#issuecomment-126360191 .

philngo commented 9 years ago

Aha! Thanks, Michael. Changed title to reflect that so we remember to fix.

mblasnik commented 9 years ago

Well, I'm still getting the error message but it's taking much longer before showing up. I confirmed that the time stamps are unique within thermostat id.

On Thu, Jul 30, 2015 at 11:12 AM, Phil Ngo notifications@github.com wrote:

Aha! Thanks, Michael. Changed title to reflect that so we remember to fix.

— Reply to this email directly or view it on GitHub https://github.com/impactlab/thermostat/issues/4#issuecomment-126366464.

philngo commented 9 years ago

Hmmm. Exact same error?

mblasnik commented 9 years ago

yep -- and I re-ran the import line to make sure it used the newer version of the csv file. Then when I re-ran the loop it actually seemed to be working or at least doing something for a while -- maybe 60 seconds -- and then it threw what looks like the same error:

Traceback (most recent call last):

File "", line 2, in

File "/usr/local/nest-python-local/lib/python/site-packages/thermostat/metrics.py", line 33, in calculate_epa_draft_rccs_field_savings_metrics

for cooling_season, season_name in thermostat.get_cooling_seasons():

File "/usr/local/nest-python-local/lib/python/site-packages/thermostat/core.py", line 349, in get_cooling_seasons

total_cooling = pd.DataFrame(cooling_columns).sum(axis=0)

File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 243, in init

arrays, columns = _to_arrays(data, columns, dtype=dtype)

File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 4748, in _to_arrays

dtype=dtype)

File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 4841, in _list_of_series_to_arrays

indexer = indexer_cache[id(index)] = index.get_indexer(columns)

File "/usr/local/nest-python-2.7.8-Darwin-x86_64-20140826/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/index.py", line 1284, in get_indexer

raise InvalidIndexError('Reindexing only valid with uniquely'

pandas.core.index.InvalidIndexError: Reindexing only valid with uniquely valued Index objects

On Thu, Jul 30, 2015 at 11:18 AM, Phil Ngo notifications@github.com wrote:

Hmmm. Exact same error?

— Reply to this email directly or view it on GitHub https://github.com/impactlab/thermostat/issues/4#issuecomment-126368183.

philngo commented 9 years ago

Nothing jumps out at me right away. Mind if I try to reproduce the error with the input files you're using?

mblasnik commented 9 years ago

what is your email?

On Thu, Jul 30, 2015 at 11:28 AM, Phil Ngo notifications@github.com wrote:

Nothing jumps out at me right away. Mind if I try to reproduce the error with the input files you're using?

— Reply to this email directly or view it on GitHub https://github.com/impactlab/thermostat/issues/4#issuecomment-126370542.

philngo commented 9 years ago

Just sent you a message from my impact lab email.

philngo commented 9 years ago

Just submitted a fix for the issue you were running into. Please let me know if you continue to run into errors.

b0a03d06b127829a6e4e3401bf3a0dc1e3f1feb5

mblasnik commented 9 years ago

Thanks -- I'll try it out tomorrow and let you know.

On Thu, Jul 30, 2015 at 5:13 PM, Phil Ngo notifications@github.com wrote:

Just submitted a fix for the issue you were running into. Please let me know if you continue to run into errors.

b0a03d0 https://github.com/impactlab/thermostat/commit/b0a03d06b127829a6e4e3401bf3a0dc1e3f1feb5

— Reply to this email directly or view it on GitHub https://github.com/impactlab/thermostat/issues/4#issuecomment-126489464.