Bears-R-Us / arkouda

Arkouda (αρκούδα): Interactive Data Analytics at Supercomputing Scale :bear:
Other
232 stars 87 forks source link

DataFrame round trip conversion discrepancies #3426

Closed ajpotts closed 1 month ago

ajpotts commented 1 month ago

Describe the bug Converting a DataFrame to pandas and back does not give the expected behavior

To Reproduce


In [48]: size = 100

In [49]: a = ak.arange(size, dtype="float64")

In [50]: idx = ak.Index(a)

In [51]: df = ak.DataFrame({"col1": a}, index=idx)

In [52]: df2 = ak.DataFrame(df.to_pandas(retain_index=True))

In [53]: df.index.inferred_type
Out[53]: 'floating'

In [54]: df2.index.inferred_type
Out[54]: 'integer'

Expected behavior Converting to pandas and back to arkouda should return a copy of the original object, with all the same dtypes and attributes.

ajpotts commented 1 month ago

After investigating the above example, to_pandas has retain_index set default to False, but there is was also a omission where DataFrame._set_index did not handle the pd.Index case.