NREL / flasc

A rich floris-driven suite for SCADA analysis
https://nrel.github.io/flasc/
BSD 3-Clause "New" or "Revised" License
31 stars 18 forks source link

Improve floris dataframe interpolation function #124

Closed Bartdoekemeijer closed 9 months ago

Bartdoekemeijer commented 1 year ago

This PR is ready to be merged.

Feature or improvement description Split interpolation into 2D and 3D grid, and general pandas improvements to replace manually adding rows with pd.concat() function. Significant speed-ups achieved.

Related issue, if one exists N/A

Impacted areas of the software floris_tools

Additional supporting information Found a significant speed-up with in-house work. Found it hard to reproduce, but see code below:

Test results, if applicable Here's a minimal example to reproduce:

import os
import numpy as np
import pandas as pd
from time import perf_counter as timerpc

from flasc.floris_tools import interpolate_floris_from_df_approx
from flasc.utilities_examples import load_floris_artificial as load_floris

if __name__ == "__main__":
    # Create fake df_fi_approx table
    wd_mesh, ws_mesh = np.meshgrid(
        np.arange(0.0, 360.0, 1.0),
        np.arange(1.0, 30.0, 1.0)
    )
    N = len(wd_mesh.flatten())
    fake_power = 1500.0 + 30.0 * np.random.randn(N, 100)
    df_fi_approx = pd.concat([
        pd.DataFrame(
            {
                "wd": wd_mesh.flatten(),
                "ws": ws_mesh.flatten(),
                "ti": 0.08 * np.ones_like(ws_mesh.flatten())}
        ),
        pd.DataFrame(fake_power, columns=[f"pow_{ti}" for ti in range(100)])
    ], axis=1)

    # Now interpolate from table
    N = int(1e8)  # Number of measurements
    df = pd.DataFrame(
        {
            "wd": 360.0 * np.random.uniform(N),
            "ws": 1.01 + 28.0 * np.random.uniform(N),
            "ti": 0.08 * np.ones(N),
        }
    )

    # Perform interpolation
    t0 = timerpc()
    df = interpolate_floris_from_df_approx(df, df_fi_approx, mirror_nans=False)
    dt = timerpc() - t0
    print(f"Completed 'interpolate_floris_from_df_approx' call in {dt:.3f} seconds.")

This times at 41 seconds with the new code, and 50 seconds in the old code. So about a 20% improvement in this simplified example. Found a factor 3-4 speed-up in my in-house codes though.

paulf81 commented 9 months ago

Yes, ok for me too to merge. Thank you for checking @misi9170 and thanks for the submission @Bartdoekemeijer