The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Today users have very little visibility when new versions of a connector are released. When we release core, we do so with a fixed set of connector versions. This isn't really a problem. The issue is that if we release a new version of a connector after the fact, the user has no good way (in the UI) to figure out that there is a new connector available.
They effectively get stuck on a connector version until they ask for help. We have numerous user issues due to connector version problems (e.g. issue). This is a common occurrence because connectors evolve at a different pace than core. It is pretty common that a user will not need upgrade their current core version but will want to upgrade a connector version to get a bug fix or new feature.
Describe the solution you’d like
I would like airbyte to be able to pull from a feed updates about connectors and detect:
there is a new version and allow to update
ensure that core is compatible with the new version of the connector. This should be less of an issue as we stabilize the protocol. (I propose we punt on this for now. It can be done on a follow on task with no additional overhead. The reason to punt is that it requires us to be more thoughtful about protocol version and we haven't figure that out yet, cgardens)
Describe the alternative you’ve considered or used
Today we have an escape hatch where if a user knows that there is a new version of a connector, they can manually input its version number in the UI. This process requires them to either read our change log or find where we declare new versions of connectors in the code base. This process is not ideal.
cgardens
commented
3 years ago
Tell us about the problem you're trying to solve
Today users have very little visibility when new versions of a connector are released. When we release core, we do so with a fixed set of connector versions. This isn't really a problem. The issue is that if we release a new version of a connector after the fact, the user has no good way (in the UI) to figure out that there is a new connector available.
They effectively get stuck on a connector version until they ask for help. We have numerous user issues due to connector version problems (e.g. issue). This is a common occurrence because connectors evolve at a different pace than core. It is pretty common that a user will not need upgrade their current core version but will want to upgrade a connector version to get a bug fix or new feature.
Describe the solution you’d like
I would like airbyte to be able to pull from a feed updates about connectors and detect:
ensure that core is compatible with the new version of the connector. This should be less of an issue as we stabilize the protocol.(I propose we punt on this for now. It can be done on a follow on task with no additional overhead. The reason to punt is that it requires us to be more thoughtful about protocol version and we haven't figure that out yet, cgardens)Describe the alternative you’ve considered or used
Today we have an escape hatch where if a user knows that there is a new version of a connector, they can manually input its version number in the UI. This process requires them to either read our change log or find where we declare new versions of connectors in the code base. This process is not ideal.
Issue for the UI work: https://github.com/airbytehq/airbyte/issues/2218