Support pandas 2.0 - Githubissues

jtilly commented 1 year ago

Our CI jobs for pandas 2.0 are currently failing (see, e.g., here).

I see (at least) two issues with supporting pandas 2.0:

https://github.com/pandas-dev/pandas/issues/50127 (open PR here): this issue is causing our CI failures
https://github.com/pandas-dev/pandas/pull/52212 changed how pandas is inferring dtypes from scalars, which results in issues when we partition by a datetime (loading the data eagerly will return a datetime64[ns], loading the data as dask data frame will return a datetime64[s]). I don't think we have tests for this by the way. That we could address by explicitly setting the units to nanoseconds here.

lbittarello commented 1 year ago

Has this repo been abandoned? Nobody is addressing this issue or even approving the automated PRs...

lbittarello commented 1 year ago

@DamianBarabonkovQC @xhochy

xhochy commented 1 year ago

@DamianBarabonkovQC @xhochy

Open for PRs.

data-engineering-collective / plateau