Change df[col].apply(lambda x: ast.literal_eval(x)) to either df[col].apply(lambda x: ast.literal_eval(x.replace('null', 'None'))) or df[col].apply(lambda x: json.loads(x)), otherwise the current code throws ValueError: malformed node or string when the dict contents of extra_resources or env_vars contain a null.
Change
df[col].apply(lambda x: ast.literal_eval(x))
to eitherdf[col].apply(lambda x: ast.literal_eval(x.replace('null', 'None')))
ordf[col].apply(lambda x: json.loads(x))
, otherwise the current code throwsValueError: malformed node or string
when the dict contents ofextra_resources
orenv_vars
contain a null.