search

chengchingwen / Pickle.jl

An experimental package for loading and saving object in Python Pickle format.
MIT License
51 stars 9 forks source link

Hope to support pandas.DataFrame object #25

Open zsz00 opened 2 years ago

zsz00 commented 2 years ago

Hope to support pandas.DataFrame object read and save DataFrames as pandas.DataFrame...

chengchingwen commented 2 years ago

To what extent do you need this feature? i.e. Do you need both reading and saving DataFrame? What set of element types would you need?

zsz00 commented 2 years ago

i need read pd.DataFrame to DataFrames.DataFrame and write DataFrames.DataFrame to pd.DataFrame in julia.

most need read pd.DataFrame to DataFrames.DataFrame in julia.

chengchingwen commented 2 years ago

What kind of value would your dataframe contain? number / string / datetime?

zsz00 commented 2 years ago

number / string /bool

zsz00 commented 2 years ago

I found an easy way to do it: https://cjdoris.github.io/PythonCall.jl/stable/compat/#PythonCall.pytable

using DataFrames, PythonCall
@py import pandas as pd

input_table = pd.read_pickle("/mnt/img_list_2.csv_2.pkl")   # read to julia obj: PyPandasDataFrame, compatible Tables.jl. 
jl_df = DataFrame(input_table)

py_df = pytable(jl_df)    # julia DataFrame to  pd.DataFrame
py_df.describe()
pd.to_pickle(py_df, "data.pkl")
chengchingwen commented 2 years ago

Yes, you can always use PyCall.jl or PythonCall.jl to call those python function directly. The purpose of this package (Pickle.jl) is to provide a pure Julia implementation so that we can avoid python dependencies.