search

kbajalc / parquets

MIT License
44 stars 21 forks source link

Improve performance of reads by accessing the records by index instead of shift() #28

Open ChrisZieba opened 3 years ago

ChrisZieba commented 3 years ago

Original PR: https://github.com/ironSource/parquetjs/pull/107 by @rickyk586.

Improve performance of reads by accessing the records by index instead of by .shift(). Improved read performance by 30x locally.

I'm also seeing very significant improvements in performance. As far as I can tell, there's no reason to be using shift here.