search

MeltanoLabs / Singer-Working-Group

Working group for ongoing development and iteration of the Singer Spec, the de-facto protocol for open source data connectors. Please use "Issues" to create discussion items - or use "Discussions" for general questions.
Apache License 2.0
13 stars 4 forks source link

Spec Proposal: `BATCH` message type (aka `FASTSYNC`) #2

Open aaronsteers opened 3 years ago

aaronsteers commented 3 years ago

As inspired by the FASTSYNC spec in Pipelinewise and further described here: https://gitlab.com/meltano/sdk/-/issues/9

Meltano also has a WIP spec discussion and issue tracking this here: https://gitlab.com/meltano/sdk/-/issues/9

louis-pie commented 3 years ago

I think this is a very important feature if someone is ever to migrate from PipelineWise to Meltano

judahrand commented 2 years ago

I think this is a very important feature if someone is ever to migrate from PipelineWise to Meltano

Absolutely, it was a shame that we could not consider Meltano due to the initial size of our tables.

hhcomply commented 3 weeks ago

Hi @aaronsteers this is a great idea! Was this feature ever implemented?

aaronsteers commented 3 weeks ago

@hhcomply re:

Hi @aaronsteers this is a great idea! Was this feature ever implemented?

Yes! We implement into the Meltano SDK: