One of the provided ErrorHandling implementations. Title is actually little misleading, point is to write the errors into string column, and the column name should default into spark.sql.columnNameOfCorruptRecord (See Runtime SQL Configuration)
Feature
Write errors into a StringType column, by converting each error submit filed into a string and concatenating them with a delimiter. The column name should/might default to spark.sql.columnNameOfCorruptRecord
Proposed Solution
Solution Ideas:
Configurable error delimiter (delimiter between different errors), default \n
Background
One of the provided
ErrorHandlingimplementations. Title is actually little misleading, point is to write the errors into string column, and the column name should default intospark.sql.columnNameOfCorruptRecord(See Runtime SQL Configuration)Feature
Write errors into a
StringTypecolumn, by converting each error submit filed into a string and concatenating them with a delimiter. The column name should/might default tospark.sql.columnNameOfCorruptRecordProposed Solution
Solution Ideas:
\n,