search

Volue-Public / energy-mesh-python

A Python API able to communicate with Volue Energy's Mesh server.
Other
10 stars 0 forks source link

read_time_series_points returns NaN at the last time step for some time series #511

Open asmunds opened 1 month ago

asmunds commented 1 month ago

Mesh Python SDK version: 1.8.0 Mesh version: Not sure how to find out (let me know if it is important) Python version: 3.11.9 Operating System (Windows, Ubuntu, etc.): Windows

When running the following script:


import datetime
import uuid
import sys
import aa_mesh_client
import pandas as pd

def main():
    tz = datetime.timezone(datetime.timedelta(hours=1))

    config = {}
    config['gyvatn'] = 132864
    config['holsmoen'] = 3087
    config['hartevatn'] = uuid.UUID('22f9398e-b59d-42b7-b4ed-9be8ee745219')
    config['botsvatn'] = uuid.UUID('69f843e2-0f67-4134-ac15-271386c37e2b')
    s_time = datetime.datetime(2024, 8, 1, 0, tzinfo=tz)
    e_time = datetime.datetime(2024, 9, 2, 2, tzinfo=tz)
    df = read_mesh_ts_tims_key(config, s_time, e_time)
    df.index = df.index + datetime.timedelta(hours=1)
    df.index.names = ['normal_time']
    print(df)

def read_mesh_ts_tims_key(config, s_time, e_time):
    mesh_client = aa_mesh_client.MeshClient.from_device_flow()
    con = mesh_client.mesh_connection  # type: mesh.Connection
    df = pd.DataFrame()

    with con.create_session() as session:
        for k, v in config.items():
            data = session.read_timeseries_points(
                target=v,
                start_time=s_time,
                end_time=e_time
            )
            tmp_df = data.arrow_table.to_pandas().set_index('utc_time')
            df[k] = tmp_df['value']

    return df

if __name__ == '__main__':
    main()

I get:

                      gyvatn  holsmoen  hartevatn  botsvatn
normal_time
2024-08-01 01:00:00  10.6439    7.6064   758.3671  532.5551
2024-08-01 02:00:00  10.7240    7.5966   758.3685  532.5612
2024-08-01 03:00:00  10.2795    7.3516   758.3701  532.5656
2024-08-01 04:00:00  10.0172    6.7281   758.3686  532.5698
2024-08-01 05:00:00  10.2586    6.3728   758.3701  532.5687
...                      ...       ...        ...       ...
2024-09-01 23:00:00   8.9891    7.1370   757.9425  542.9951
2024-09-02 00:00:00   9.1229    6.6700   757.9359  542.9871
2024-09-02 01:00:00   9.5427    6.3853   757.9244  542.9808
2024-09-02 02:00:00   9.4680    6.1964   757.9182  542.9695
2024-09-02 03:00:00   9.1774       NaN        NaN       NaN

If i increase e_time by one hour, I get

                      gyvatn  holsmoen  hartevatn  botsvatn
normal_time
2024-08-01 00:00:00  10.9156    7.9770   758.3683  532.5478
2024-08-01 01:00:00  10.6439    7.6064   758.3671  532.5551
2024-08-01 02:00:00  10.7240    7.5966   758.3685  532.5612
2024-08-01 03:00:00  10.2795    7.3516   758.3701  532.5656
2024-08-01 04:00:00  10.0172    6.7281   758.3686  532.5698
...                      ...       ...        ...       ...
2024-09-02 00:00:00   9.1229    6.6700   757.9359  542.9871
2024-09-02 01:00:00   9.5427    6.3853   757.9244  542.9808
2024-09-02 02:00:00   9.4680    6.1964   757.9182  542.9695
2024-09-02 03:00:00   9.1774    5.6755   757.9091  542.9630
2024-09-02 04:00:00   8.7272       NaN        NaN       NaN

which goes to show that there are actually no NaNs in the data at these time stamps.

This seems to me like a bug.

erny-powel commented 1 month ago

We'll have a look.

Mesh version: Not sure how to find out (let me know if it is important)

Mostly convenient as it saves us from a bit of investigation. In Mesh Python the mesh.Connection class has a get_version method that you can use, see https://github.com/Volue-Public/energy-mesh-python/blob/master/src/volue/mesh/examples/connect_synchronously.py.

asmunds commented 1 month ago

VersionInfo(version='2.14.2+19', name='Volue Mesh Server')