search

functime-org / functime

Time-series machine learning at scale. Built with Polars for embarrassingly parallel feature extraction and forecasts on panel data.
https://docs.functime.ai
Apache License 2.0
1.02k stars 55 forks source link

Error in transform_new Function Due to Index Column Mismatch in Scale Transformer #227

Open jhug12 opened 3 months ago

jhug12 commented 3 months ago

Describe the bug The transform_new function of the scale transformer in the functime preprocessing module fails during execution. This issue arises when the function attempts to join the X DataFrame with the _mean DataFrame based on index columns (idx_cols) specified as 'time' and 'entity'. The join operation fails because _mean contains only the 'entity' column and lacks the 'time' column, leading to a key mismatch and an execution error.

To Reproduce

 import polars as pl
 from functime.preprocessing import scale

 # Load sample data
 y = pl.read_parquet("https://github.com/functime-org/functime/raw/main/data/commodities.parquet")

 # Initialize the scale transformer
 transformer = scale()

 # Fit the transformer
 _ = y.pipe(transformer).collect()

 # Attempt to transform new data and observe the error
 transformer.transform_new(y).collect()

Expected behavior The function would only join on columns that are present in _mean.

Desktop (please complete the following information):

baggiponte commented 3 months ago

Ciao and thanks for the bug report! Would also like ask you what Polars version you are running.