Closed imrimt closed 8 months ago
In fact, as I typed out the description, I figured out my own workaround:
SaveMode.Append
"preactions": "TRUNCATE TABLE IF EXISTS %s;"
truncate_table
Will close this issue but hopefully it'll be helpful for others who might have similar use cases.
We have a use case where we want the following:
SaveMode.Overwrite
truncate_table: on
because we don't want the table to be dropped (there are grants and shares that we want to keep)column_map
is specifiedHowever, it looks like
column_map
is not supported inoverwrite
mode.How should we go about this use case? It feels like an oversight as
truncate_table: on
is essentially the same asSaveMode.Append
with the additional step of clearing out the data.We are currently using
2.11.3-spark_3.1
connector.