ParquetReader ignored read_columns argument

astronomy-commons / hipscat-import

HiPSCat import - generate HiPSCat-partitioned catalogs

https://hipscat-import.readthedocs.io

BSD 3-Clause "New" or "Revised" License

5 stars 2 forks source link

Closed delucchi-cmu closed 1 month ago

delucchi-cmu commented 2 months ago

Bug report

This is a problem for two reasons:

reads too many columns during map phase, as we only need ra/dec.
if user just wanted to read a few columns of their data to import, this argument would not be passed along to kwargs