A minimal implementation for the tap to accept the metadata columns _sdc_source_file and _sdc_source_lineno as unique keys in the key_properties config.
This enables pipelines that use file names as part of the unique key, and makes it possible to set up pipelines without knowing the unique key of the data and still, in most real-world use cases [citation needded], get it right.
Includes a simple test for this use case.
As a sidenote, because of the header being line one, no line number 1 will turn up in the target data.
A minimal implementation for the tap to accept the metadata columns
_sdc_source_fileand_sdc_source_linenoas unique keys in thekey_propertiesconfig.This enables pipelines that use file names as part of the unique key, and makes it possible to set up pipelines without knowing the unique key of the data and still, in most real-world use cases [citation needded], get it right.
Includes a simple test for this use case.
As a sidenote, because of the header being line one, no line number 1 will turn up in the target data.