search

sktime / sktime

A unified framework for machine learning with time series
https://www.sktime.net
BSD 3-Clause "New" or "Revised" License
7.74k stars 1.32k forks source link

[ENH] Adding PandasDataset from GluonTS support in Sktime #6658

Closed shlok191 closed 1 month ago

shlok191 commented 2 months ago

Is your feature request related to a problem? Please describe. I am currently working on implementing an interface for LagLlama - link here - and one of the accepted input types for LagLlama is GluonTS' PandasDataset.

Describe the solution you'd like I wish to add support for the PandasDataset object type in sktime. I have already worked on GluonTS in sktime, and am quite familiar with both frameworks.

Describe alternatives you've considered An alternate solution would be using CSV files, but since PandasDataset offer easy conversions from pd.DataFrame, I think this implementation will be really helpful.

Additional context Nothing additional to add for now.

shlok191 commented 2 months ago

@fkiraly, beginning work on this! :)

fkiraly commented 2 months ago

great.

Can you link the spec for PandasDataset? How is it different from pandas native containers?

shlok191 commented 2 months ago

From what I see here, https://ts.gluon.ai/stable/tutorials/data_manipulation/pandasdataframes.html

They have not specified an explicit reason as to why they do not use the standard pd.DataFrame type. I suspect it is because the GluonTS PandasDataset must follow a certain schema for organizing the data, where each unique time series must have it's own item_id and a common timestamp value.

I think it makes sense to enforce their own formatting, which is likely why they have this wrapper

fkiraly commented 2 months ago

I think the abstract challenge is "how to represent Panel data".

gluonts is taking the common approach of coming up with their own data container - sktime goes the opposite way and uses existing data containers that are commonly used, and manages specifications, e.g., pd-multiindex is pandas with specific conventions.