Spec Proposal: `BATCH` message type (aka `FASTSYNC`)

MeltanoLabs / Singer-Working-Group

Working group for ongoing development and iteration of the Singer Spec, the de-facto protocol for open source data connectors. Please use "Issues" to create discussion items - or use "Discussions" for general questions.

Apache License 2.0

13 stars 4 forks source link

Spec Proposal: `BATCH` message type (aka `FASTSYNC`) #2

Open aaronsteers opened 2 years ago

aaronsteers commented 2 years ago

As inspired by the FASTSYNC spec in Pipelinewise and further described here: https://gitlab.com/meltano/sdk/-/issues/9

Meltano also has a WIP spec discussion and issue tracking this here: https://gitlab.com/meltano/sdk/-/issues/9

louis-pie commented 2 years ago

I think this is a very important feature if someone is ever to migrate from PipelineWise to Meltano

judahrand commented 2 years ago

I think this is a very important feature if someone is ever to migrate from PipelineWise to Meltano

Absolutely, it was a shame that we could not consider Meltano due to the initial size of our tables.