Closed AnIrishDuck closed 1 year ago
Patch coverage: 100.00
% and project coverage change: +0.01
:tada:
Comparison is base (
3ab9b61
) 83.45% compared to head (b8a4882
) 83.46%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
Thanks for the PR @AnIrishDuck. What was the behavior we had first that was flawed? For my understanding of the changes here.
When trying to decode pandas-record json with list arrays, it would expect them to be nested within another list array. In other words, deserialization of ListArray
s for pandas record JSON was pretty much generally broken.
We missed this on implementation because we were using FixedSizeList
in most places internally.
To elaborate (after looking at the tests), we only tested that nested list arrays worked, which hid the underlying issue (as the outer level of nesting that infer_schema
would generate would still be properly handled by allocate_array
)
Right, Thanks!
The double-list construction was a mistake that prevents parsing of several valid types of pandas record json constructs.
Fix that by removing the nested lists and add a test verifying that it works.