use polars lazy api

[ritchie46]

I see you are running eager polars. E.g. using read_parquet. To do a fair comparison you must scan_parquet, write your query lazily and collect at the end.

Now duckdb can use query optimization, whilst polars cannot because you force every operation to execute immediately. That's not an apples vs apples comparison.

[prrao87]

Thanks @ritchie46, that's a good point and I was questioning myself about this just yesterday! 😀

I'll switch to the lazy API and rerun.