timescale / timescaledb-toolkit

Extension for more hyperfunctions, fully compatible with TimescaleDB and PostgreSQL 📈
https://www.timescale.com
Other
378 stars 47 forks source link

A two-step fast Fourier transform aggregate functions #630

Open tucnak opened 1 year ago

tucnak commented 1 year ago

Is your feature request related to a problem? Please describe.

Sometimes, it useful to think of time series data in the frequency domain; this is especially important in areas like fraud detection, or feature engineering in the world of machine learning. In some tasks, frequency domain features are more instructive to reason about compared to time domain signals where you sometimes would be more exposed to variance.

TimescaleDB is a time series database, so it would make sense to include something like this.

I mean, there's histogram, and I would bet that there are good FFT implementations in Rust, so in principle, it could be made into a custom aggregate— potentially even PARALLEL SAFE kind— to be used with continuous aggregates in your scale-out model, which would certainly make some people happy.

Describe the solution you'd like

The aggregate function fft_agg() not unlike histogram() but a little bit more involved parameter-wise, and perhaps two-step, so it could be used easily in a continuous aggregate and potentially even be rolled up so I can tag my frequency domain representation along with my trusty old stats, percentiles.

Describe alternatives you've considered

The alternative is to calculate this outside Postgres but we want this for feature engineering to be used with PostgresML.

Best regards,

gegnew commented 9 months ago

would also love to see this!