Describe the bug
queries such as read_parquet().count_rows() should not do a full scan, and instead be should able to be fulfilled by the metadata only.
The need for the full scan should be optimized away during physical planning.
To Reproduce
Steps to reproduce the behavior:
daft.read_parquet().count_rows() **Expected behavior** Metadata only operations such ascount(*)orcount_rows()` usually can be fulfilled without needing to perform a full scan.
Additional context
dump from daft.read_parquet('lineitem.parquet').explain(show_all=True)
Describe the bug queries such as
read_parquet().count_rows()
should not do a full scan, and instead be should able to be fulfilled by the metadata only.The need for the full scan should be optimized away during physical planning.
To Reproduce Steps to reproduce the behavior:
daft.read_parquet().count_rows() **Expected behavior** Metadata only operations such as
count(*)or
count_rows()` usually can be fulfilled without needing to perform a full scan.Additional context dump from
daft.read_parquet('lineitem.parquet').explain(show_all=True)