Switching the dataloader build_entry_list from pandas to polars

legend-exp / pygama

Python package for data processing and analysis

https://pygama.readthedocs.io

GNU General Public License v3.0

18 stars 56 forks source link

Switching the dataloader build_entry_list from pandas to polars #527

Open ggmarshall opened 12 months ago

ggmarshall commented 12 months ago

seems to give a decent speedup in testing, definitely more optimisation possible as well with using Lazy dataframes. have left the cut query as pandas as this needs a parser to work with polars

gipert commented 12 months ago

Jesus... what's this commit history?

ggmarshall commented 12 months ago

I see no problem.....

ggmarshall commented 12 months ago

It's because of the rebasing, will leave as this is just an exemplar