Open maarten-van-wauwe opened 1 year ago
[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of pandas.
[X] I have confirmed this bug exists on the main branch of pandas.
import pandas as pd import json json_text = """ {"root": [{ "id":1, "field1":"apple", "field2":"banana", "siblings":[ {"sibling_id": "1", "sibling1": "horse", "sibling2": "ape", "subsiblings": [{"a": "b"}] }] }, { "id":2, "field1":"cola", "field2":"dwarf", "siblings":null }] } """ json_object = json.loads(json_text) print(pd.json_normalize(json_object, record_path=['root'])) print(pd.json_normalize(json_object, record_path=['root','siblings'], meta=[['root','id']])) print(pd.json_normalize(json_object, record_path=['root','siblings','subsiblings'], meta=[['root','siblings','sibling_id']]))
element with id = 2 has NULL in siblings field. This results in a TypeError: 'NoneType' object is not iterable
Solution: add following 2 lines to the definition of the "_recursive_extract" function
def _recursive_extract(data, path, seen_meta, level=0): if data is None: return
null elements should be skipped for further recursive processing
this failed
this happened with several API's that returned me JSON. eg: Skype json that you can download as backup of conversations
Hi! I can work on this issue if it is not already being worked on
Pandas version checks
[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of pandas.
[X] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
element with id = 2 has NULL in siblings field. This results in a TypeError: 'NoneType' object is not iterable
Solution: add following 2 lines to the definition of the "_recursive_extract" function
Expected Behavior
null elements should be skipped for further recursive processing
Installed Versions
this failed