Dynamic loading of sensor data

[JrtPec]

cfr. databases, where you get the option of querying data "lazy" or "dynamic"

Right now, a call to hp.get_data() is lazy: it creates a big list of Data Series per sensor and concatenates them. This can take a while, specifically when you need many sensors and a large time span.

Often, you don't use the big DataFrame as a whole, you rather re-divide it by using a for loop and looking at each sensor individually.

I propose to add a flag hp.get_data(dynamic=True), where the return type is not a DataFrame, but rather a dynamically generated list of Series.

Would look something like this:

series = []
for sensor in sensors:
    s = sensor.get_data(head=head, tail=tail, diff=diff, resample=resample, unit=unit)
    if dynamic:
        yield s
    else:
        series.append(s)

if not dynamic:
    df = pd.concat(series, axis=1)
    return df

[JrtPec]

This is impossible: you cannot yield and return in the same method. I think I will just define a separate generator then: hp.get_data_dynamic or something

[saroele]

Seems to be merged now, I'll close this. Thanks :-)