ranaroussi / pystore

Fast data store for Pandas time-series data
Apache License 2.0
556 stars 99 forks source link

drop fastparquet and use pyarrow. this is required on latest versions of dask #70

Open gnzsnz opened 5 months ago

gnzsnz commented 5 months ago

drop default engine fastparquet and implement pyarrow

latest versions of dask use pyarrow as default engine. pandas is heading in the same direction.

gnzsnz commented 5 months ago

pull request #71 created

gnzsnz commented 4 months ago

After a series of commits PR #71 provides a working version of pystore.

In it's current state master branch and latest pipy package are not usable (at least not with latest versions of parquet and dask). at the very least an update on requirements.txt would be needed with a restriction on pandas and dask version. I have not identified what would be the right combination.

This PR has 2 problems:

besides that pystore is now working.

@ranaroussi please review this PR

@ranaroussi if you need help maintaining this package. I offer as volunteer.

gnzsnz commented 4 months ago

fix for multi-threading applied on f8e2dfd

still making my mind if this is THE fix, or just a step on the right direction

@ranaroussi I would appreciate your comments

gnzsnz commented 3 months ago

any update on this?