feat: support `pd.Series.explode` with `ExtensionArray._explode` method

intake / akimbo

For when your data won't fit in your dataframe

https://akimbo.readthedocs.io

BSD 3-Clause "New" or "Revised" License

30 stars 6 forks source link

feat: support `pd.Series.explode` with `ExtensionArray._explode` method #46

Closed douglasdavis closed 5 months ago

douglasdavis commented 8 months ago

Fix for #38

Now that pandas 2.2.0 is out we can implement this

CrfzdPQM6 commented 6 months ago

@martindurant suggested I paste this snippet here as a useful test:

import awkward as ak
values = [ak.Array([1.0,2.0,3.0]), ak.Array([4.0,5.0]), ak.Array([6.0])]
df = pd.DataFrame({'pt':values})

because the output of df.dtypes is that pt is object

I'm running into issues with pandas 2.2.1 that df.explode() does not work for a column with dtype awkward

martindurant commented 6 months ago

df = pd.DataFrame({'pt':values})

needs to have dtype specified (or astype called) to make it awkward, else you are simply calling pandas' default iteration, which would also work for python lists.