opengridcc / opengrid-dev

Open source building monitoring, analysis and control
Apache License 2.0
26 stars 21 forks source link

Removed pandas 0.18.1 workaround, gave errors with newer versions. #171

Closed GMathyssen closed 7 years ago

GMathyssen commented 7 years ago

Installed pandas 0.19.1, the workaround code gave an error because the .concat bug is fixed from version 0.18.1+

coveralls commented 7 years ago

Coverage Status

Coverage increased (+0.1%) to 70.688% when pulling ecbbf8c41f25e110e4b203b668bb7f566accd5f6 on GMathyssen:pandas_upgrade into 0b2a15ec15ed912a20870e89dfb7187336970186 on opengridcc:develop.

saroele commented 7 years ago

Do you remember the bug? I would be surprised if the empty attribute would not exist anymore, and I don´t see how the code can generate other bugs?

GMathyssen commented 7 years ago

This it what I get with pandas 0.19.1:


ValueError Traceback (most recent call last)

in () 4 endtime = dt.time(5, tzinfo=BXL) 5 caching.cache_results(hp=hp, sensors=sensors, resultname='elec_min_night_0-5', AnalysisClass=DailyAgg, ----> 6 agg='min', starttime=starttime, endtime=endtime) 7 8 caching.cache_results(hp=hp, sensors=sensors, resultname='elec_max_night_0-5', AnalysisClass=DailyAgg, /Users/Gert/opengrid/opengrid/library/caching.pyc in cache_results(hp, sensors, resultname, AnalysisClass, **kwargs) 326 327 # get new data, full resolution --> 328 df_new = hp.get_data(sensors = [sensor], head=last_day) 329 330 # apply the method /Users/Gert/opengrid/opengrid/library/houseprint/houseprint.py in get_data(self, sensors, sensortype, head, tail, diff, resample, unit) 476 series = [s for s in series if not s.empty] 477 --> 478 df = pd.concat(series, axis=1) 479 480 # Add unit as string to each series in the df. This is not persistent: the attribute unit will get /Users/Gert/og/lib/python2.7/site-packages/pandas/tools/merge.pyc in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, copy) 1332 keys=keys, levels=levels, names=names, 1333 verify_integrity=verify_integrity, -> 1334 copy=copy) 1335 return op.get_result() 1336 /Users/Gert/og/lib/python2.7/site-packages/pandas/tools/merge.pyc in __init__(self, objs, axis, join, join_axes, keys, levels, names, ignore_index, verify_integrity, copy) 1365 1366 if len(objs) == 0: -> 1367 raise ValueError('No objects to concatenate') 1368 1369 if keys is None: ValueError: No objects to concatenate
saroele commented 7 years ago

Ok, so this is the same error message as #170 . I solved it in that pull request by checking if series is an empty list or not, you solved it by upgrading pandas.
Requiring the last pandas version sounds rather strict to me, most users will not switch so fast. OTOH, we can force them to upgrade in the next OpenGrid release. @JrtPec what is the pandas version on azure?

I suppose my solution also works with pandas 0.19.1 Let´s hold this pull request and apply it a bit later?

GMathyssen commented 7 years ago

Ok, true

JrtPec commented 7 years ago

@saroele The current Pandas version on Azure Machine Learning is 0.18.0, so it would be nice if we'd develop code that is compatible with that version.

saroele commented 7 years ago

@GMathyssen I propose to close this issue as it was solved in a different way.