Is there a way to get a `DataFrameModel` from existing `pandas.DataFrame`?

[Felix-neko]

Hi folks! And thank you for your project.

I want to use pandera to validate input dataframes of data transformation functions I have to write -- and I have a question.

Is it possible to generate a DataFrameModel from existing pandas.DataFrame?

Something like infer_schema(...), but infer_model(...)?

Is it possible?

[westurner]

• [ ] ENH,SCH: Function that maps from (dataframelib, column datatype) to (pandera, column datatype)?

• Pandas DataFrame column datatypes otoh:

  • python primitives
  • objects: object(), datetime.datetime
  • numpy dtypes like float64
  • https://numpy.org/doc/stable/reference/arrays.dtypes.html#attributes
  • numpy dtypes like ndarray and matrix
  • numpy ufuncs: https://numpy.org/doc/stable/user/basics.ufuncs.html#overriding-ufunc-behavior
  • units
  • pint https://pypi.org/project/pint (ufuncs iirc)
    • https://github.com/hgrecco/pint-pandas pandas.DataFrame(,dtype="pint[lbf ft]")
  • sympy.physics.units https://pypi.org/project/sympy
  • astropy.units https://pypi.org/project/astropy
  • unyt https://pypi.org/project/unyt
  • error propagation:
  • uncertainties: https://pypi.org/project/uncertainties/
  • sigfig: https://pypi.org/project/sigfig/
  • money types:
  • decimal.Decimal: https://docs.python.org/3/library/decimal.html
  • py-moneyed:
  • pandas types:
  • https://pandas.pydata.org/docs/user_guide/basics.html#basics-dtypes
  • https://pandas.pydata.org/docs/reference/arrays.html#pandas-arrays-scalars-and-data-types
  • https://arrow.apache.org/docs/python/generated/pyarrow.DataType.html#pyarrow.DataType.to_pandas_dtype
  • tensors,:
  • https://pytorch.org/docs/stable/tensors.html#data-types
  • https://pytorch.org/docs/stable/tensors.html#torch.Tensor
  • https://pytorch.org/torcharrow/beta/column.html#data-types
  • https://pytorch.org/torcharrow/beta/generated/torcharrow.DataFrame.to_tensor.html

[kykyi]

@Felix-neko I don't think this is possible atm, just to check, would this be to programatically ensure different dataframes follow the same schema, or would this be to get a dataframe, extract the schema, and save it to a file?