Change df[col].apply(lambda x: ast.literal_eval(x)) to either df[col].apply(lambda x: ast.literal_eval(x.replace('null', 'None'))) or df[col].apply(lambda x: json.loads(x)), otherwise the current code throws ValueError: malformed node or string when the dict contents of extra_resources or env_vars contain a null.
Change
df[col].apply(lambda x: ast.literal_eval(x))to eitherdf[col].apply(lambda x: ast.literal_eval(x.replace('null', 'None')))ordf[col].apply(lambda x: json.loads(x)), otherwise the current code throwsValueError: malformed node or stringwhen the dict contents ofextra_resourcesorenv_varscontain a null.