search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.18k stars 4.14k forks source link

[EPIC] Completely deprecate `authSpecification` usage from `airbyte protocol` #15019

Closed bazarnov closed 1 year ago

bazarnov commented 2 years ago

Currently we have 2 versions of how the source/destination could support OAuth2.0 flow implementation authSpecification and advancedAuth. We need to completely deprecate the first one and leave the last one for future support and development.

Predicted action items: 1) get the list of the connectors support authSpecification (both source/destination) 2) migrate each of them to support new advancedAuth 3) remove authSpecification from airbyte protocol without breaking anything else.

timroes commented 2 years ago

Having both authentication of those authentication methods to deal with in the Frontend currently leads to some severe tech-debt and leads ever so often to bugs like https://github.com/airbytehq/airbyte/issues/17753.

What could we do to prioritize this, so that we can remove support for authSpecification? cc @bleonard @sherifnada

sherifnada commented 2 years ago

we need to:

  1. migrate connectors off the legacy auth spec
  2. remove it from the Airbyte protocol
sherifnada commented 2 years ago

@YowanR would it be possible to have GL do step 1? extensibility can do step 2

YowanR commented 2 years ago

@sherifnada Concretely, what does Step#1 involve? I'd like to understand how much time it would take so that we can plan and prioritize accordingly

sherifnada commented 2 years ago

@YowanR It would involve:

  1. identifying which connectors still use the old spec. Done below.
  2. edit them to use the new spec (most likely a manual process)
  3. publish the changed connectors
  4. frontend merges the change to remove all usages of this logic
  5. Delete the field from the Airbyte protocol

Connectors which use the old specification: 

destination-google-sheets
source-asana
source-drift
source-facebook-marketing
source-facebook-pages
source-google-ads
source-google-analytics-data-api
source-google-analytics-v4
source-google-directory
source-google-search-console
source-google-sheets
source-harvest
source-hubspot
source-instagram
source-intercom
source-lever-hiring
source-linkedin-ads
source-linkedin-pages
source-notion
source-pipedrive
source-qualaroo
source-slack
source-snapchat-marketing
source-square
source-surveymonkey
source-trello
source-youtube-analytics
YowanR commented 2 years ago

Thanks @sherifnada! @lazebnyi @YuliiaNahoha Can you take a look at this? We should be able to fit this in our next sprint. Thoughts?

lmossman commented 1 year ago

@bazarnov @bmcgilli @lazebnyi it looks like two connectors slipped through the cracks here: source-rd-station-marketing and destination-google-sheets

Can you prioritize migrating these from authSpecification to advanced_auth, so that we can unblock deprecating this from the airbyte protocol?

lazebnyi commented 1 year ago

@lmossman Thank you for pointing that out! We will incorporate those two connectors into our plans for Q3a.

bmcgilli commented 1 year ago

@lazebnyi looking at what's left to be done for q2b, I think the team should be able to get to this next week, no? A number of items are blocked at the unblockers/contributors

sherifnada commented 1 year ago

@lazebnyi this should be done as part of q2b (we have one more week in the cycle)

lazebnyi commented 1 year ago

@lmossman Migration from authSpecification to advanced_auth for source-rd-station-marketing and destination-google-sheets has been successfully completed.