ydataai / ydata-profiling

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
https://docs.profiling.ydata.ai
MIT License
12.51k stars 1.68k forks source link

fix: fix warning related to categories #1644

Open dheenadhayalan-r opened 2 months ago

dheenadhayalan-r commented 2 months ago

Current Behaviour

typeset.py:125: DeprecationWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, pd.CategoricalDtype) instead

is_valid_dtype = pdt.is_categorical_dtype(series) and not pdt.is_bool_dtype( /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/ydata_profiling/model/pandas/summary_pandas.py:39: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set pd.set_option('future.no_silent_downcasting', True) series = series.fillna(np.nan)

Expected Behaviour

ydata profiling , unable to categorize like"Lowe Case","Upper Case" etc.

Data Description

same issue for all types of data

Code that reproduces the bug

No response

pandas-profiling version

Version: 4.9.0

Dependencies

pandas==2.2.2

OS

Databricks Runtime

Checklist

fabclmnt commented 2 months ago

Hi @dheenadhayalan-r ,

thank you for your report. Nevertheless, it is quite hard to grasp what is the issue? If our team have understood correctly it seems like it is related with solving the latest warnings issues. Can you please confirm?

Not sure what do you mean by: "Expected Behaviour

ydata profiling , unable to categorize like"Lowe Case","Upper Case" etc."

Can you please provide a more detailed report?

timschott commented 1 day ago

the issue is related to the pandas future warning. the fix is placing with pd.option_context("future.no_silent_downcasting", True): here (reference - ydata_profiling/model/pandas/summary_pandas.py:39) can you all please update this?