NREL / infrasys

Data store for components and time series in support of Python-based modeling packages
https://nrel.github.io/infrasys/
BSD 3-Clause "New" or "Revised" License
5 stars 2 forks source link

Store time series data more efficiently #11

Open daniel-thom opened 7 months ago

daniel-thom commented 7 months ago

Problems with the current design of time series:

  1. Does not allow one component to share time series data across two fields (e.g., active power and reactive power).
  2. Does not allow one component to share time series data with different initial times.
  3. The user can accidentally store the same time array data multiple times.

Dan and Pedro brainstormed a solution.

Create this new class:

class ProfileData(BaseModel):
    data: pa.Array | BaseQuantity
    hash_str: str

SingleTimeSeries would store it instead of the same data type above. The profile data would get serialized with its hash instead of the currently-used UUID. Duplicate data would never be stored.

The user does not have to know about the ProfileData class. It can get created automatically in System.add_time_series. The user interfaces stay the same. There is one new user pattern to share time series:

# ts1 is a SingleTimeSeries that is already stored in the system.
ts2 = SingleTimeSeries.from_existing(ts1, variable_name=X, initial_time=Y, resolution=Z)