search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.02k stars 4.11k forks source link

Clarify in docs relationship to Singer project from Stitch/Talend? #445

Closed alexanderdean closed 3 months ago

alexanderdean commented 4 years ago

Hi - Airbyte looks like a cool project!

I was trying to find the integrations library and it seemed empty with references to the most-excellent Singer project from Stitch/Talend.

Can you guys clarify in your documentation the relationship/dependency between Stitch and Airbyte?

┆Issue is synchronized with this Asana task by Unito

alexanderdean commented 4 years ago

I think a good comparator is the Meltano project from GitLab which is very clear on its Singer dependencies: https://meltano.com/

michel-tricot commented 4 years ago

Thanks for the feedback.

Singer does a good job at creating standalone connectors (although the quality varies a lot). We focus on creating a company wide integration platform. We are using some singer connectors to bootstrap our project but we’re still figuring out what place it should have over the long term.

We will likely maintain some sort of compatibility with their protocol so they can be orchestrated and be usable out-of-the-box in Airbyte.

alexanderdean commented 4 years ago

Thanks @michel-tricot - makes a lot of sense. Certainly no harm in bootstrapping - we used the Piwik (now Matomo) JavaScript Tracker to bootstrap Snowplow back in 2012.

The idea of generalizing the Singer protocol so that multiple projects could use it and potentially connector authors could go polyglot is super-interesting...

cgardens commented 3 months ago

Closing as no longer relevant. We do not rely on Singer at all anymore.