Closed aw-west-defra closed 2 years ago
When defining a schema I used Spark.sql.types.BooleanType with this error: PythonException: 'KeyError: BooleanType'. Full traceback below:
Spark.sql.types.BooleanType
PythonException: 'KeyError: BooleanType'. Full traceback below:
IntegerType also fails: PythonException: 'ValueError: Cannot convert non-finite values (NA or inf) to integer'. Full traceback below:
IntegerType
PythonException: 'ValueError: Cannot convert non-finite values (NA or inf) to integer'. Full traceback below:
FloatType passes: But returns [1, null] rather than [True, False].
FloatType
When defining a schema I used
Spark.sql.types.BooleanTypewith this error:PythonException: 'KeyError: BooleanType'. Full traceback below:IntegerTypealso fails:PythonException: 'ValueError: Cannot convert non-finite values (NA or inf) to integer'. Full traceback below:FloatTypepasses: But returns [1, null] rather than [True, False].