Prefilter the tablechunks if there are relevant where statements

davidnx / baby-kusto-csharp

A self-contained execution engine for the Kusto Query Language (KQL) written in C#

MIT License

26 stars 6 forks source link

Prefilter the tablechunks if there are relevant where statements #3

Open lothar7 opened 8 months ago

lothar7 commented 8 months ago

I am looking to process very large timeseries tables by using the ITableSource. They all have a timestamp column that can be used for filtering. However this wont work very well now since all the data must be scanned regardless.

If we could prune the tablechunks by time assuming there is a "where" statement in the query then we would only need to process there relevant table chunks.