Clarification on exactly-once guarantee

Azure / azure-sqldb-spark

This project provides a client library that allows Azure SQL DB or SQL Server to act as an input source or output sink for Spark jobs.

MIT License

75 stars 52 forks source link

Clarification on exactly-once guarantee #90

Closed jdocampo closed 4 years ago

jdocampo commented 4 years ago

Hi,

Seeing the docs, its not exactly clear if the single and bulk modes support exactly once guarantees when appending to a table. Could you clarify if it is possible to achieve this (I'm specially interested in the bulk scenario), and if any specific configuration is needed to achieve it?

arvindshmicrosoft commented 4 years ago

This connector does not offer such a guarantee. The newer connector does offer a NO_DUPLICATES mode which you are looking for.

Closing this issue as there are no plans to further develop this connector.