DEPR: `keep_default_dates` and `convert_dates` in `read_json`

pandas-dev / pandas

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

https://pandas.pydata.org

BSD 3-Clause "New" or "Revised" License

43.86k stars 18.01k forks source link

DEPR: `keep_default_dates` and `convert_dates` in `read_json` #59161

Open mroeschke opened 4 months ago

mroeschke commented 4 months ago

read_json has convert_dates, keep_default_dates and dtypes that all try to perform dtype conversions on the result from the ujson parser. Conflicts can arise when a column matching the specification of keep_default_dates matches a column specified in dtypes.

The proposal is to deprecate keep_default_dates and convert_dates and only use dtype's current definition of

True: attempt dtype conversion on all column (and refactor to use infer_dtype instead of astype guessing)
False: perform no dtype conversion
dict: dict keys that match column labels will be cast to type in the dict value

Animesh-Shukla commented 4 months ago

taking this issue

phershbe commented 4 months ago

@Animesh-Shukla are you still working on this? Your take was from a few weeks ago so I think that I am going to take this since I contributed on here a while ago and want to start again. If you still want it or want to work on it together, let me know!

Animesh-Shukla commented 4 months ago

you can take it @phershbe

phershbe commented 3 months ago

take

phershbe commented 3 months ago

I have most of a draft pull request mentioned here, but some tests are not passing so I am working on identifying the reasons and fixing them

phershbe commented 3 months ago

I created a pull request but have struggled with this issue, I don't want to keep in blocked so I unassigned myself in case anybody else would like to work on it.