airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.75k stars 4.04k forks source link

Remove `additionalProperties: false` from all non-JDBC connectors #14718

Closed VitaliiMaltsev closed 2 years ago

VitaliiMaltsev commented 2 years ago

Problem

As discovered in https://github.com/airbytehq/oncall/issues/289, a spec declaring "additionalProperties": false introduces the risk of accidental breaking changes. Specifically, when removing a property from the spec, existing connector configs will no longer be valid.

To make these upgrades/rollbacks smoother, we need to remove additionalProperties from connectors affected by these changes.

Action items

alafanechere commented 2 years ago

Here's the list of source connectors having additionalProperties: false in their specs:

source_name status
source-amplitude generally_available
source-stripe generally_available
source-google-search-console beta
source-notion beta
source-postgres beta
source-snapchat-marketing beta
source-airtable alpha
source-amazon-sqs alpha
source-apify-dataset alpha
source-appstore-singer alpha
source-azure-table alpha
source-bamboo-hr alpha
source-bigcommerce alpha
source-chargebee alpha
source-chargify alpha
source-chartmogul alpha
source-clickhouse alpha
source-close-com alpha
source-commercetools alpha
source-confluence alpha
farosai/airbyte-customer-io-source alpha
source-delighted alpha
source-dixa alpha
source-dockerhub alpha
source-e2e-test alpha
source-exchange-rates alpha
source-faker alpha
source-file alpha
source-freshdesk alpha
source-freshsales alpha
source-freshservice alpha
source-gitlab alpha
source-google-analytics-data-api alpha
source-google-workspace-admin-reports alpha
source-greenhouse alpha
farosai/airbyte-harness-source alpha
source-harvest alpha
source-hellobaton alpha
source-iterable alpha
farosai/airbyte-jenkins-source alpha
source-kafka alpha
source-kyriba alpha
source-lemlist alpha
source-linnworks alpha
source-looker alpha
source-microsoft-teams alpha
source-mongodb-v2 alpha
source-my-hours alpha
source-onesignal alpha
source-openweather alpha
source-orb alpha
source-orbit alpha
source-outreach alpha
farosai/airbyte-pagerduty-source alpha
source-paystack alpha
source-persistiq alpha
source-pivotal-tracker alpha
source-plaid alpha
source-pokeapi alpha
source-posthog alpha
source-prestashop alpha
source-quickbooks-singer alpha
source-recharge alpha
source-recurly alpha
source-retently alpha
source-rki-covid alpha
source-salesloft alpha
source-sendgrid alpha
source-shortio alpha
source-strava alpha
source-talkdesk-explore alpha
source-tempo alpha
source-tidb alpha
source-timely alpha
source-twilio alpha
source-us-census alpha
source-youtube-analytics alpha
source-webflow alpha
source-zenloop alpha
source-sentry alpha
source-zoom-singer alpha
source-zoho-crm alpha
source-firebolt alpha
source-elasticsearch alpha
alafanechere commented 2 years ago

Here's the list of destination connectors having additionalProperties: false in their specs:

source_name status
destination-azure-blob-storage alpha
destination-amazon-sqs alpha
destination-aws-datalake alpha
destination-keen alpha
destination-dynamodb alpha
destination-elasticsearch alpha
destination-firebolt alpha
destination-firestore alpha
destination-csv alpha
destination-local-json alpha
destination-mongodb alpha
destination-rabbitmq alpha
destination-redis alpha
destination-rockset alpha
destination-sftp-json alpha
ghcr.io/devmate-cloud/streamr-airbyte-connectors alpha
destination-google-sheets alpha
alafanechere commented 2 years ago
Here's the list of source connectors having additionalProperties: false in their schemas: source_name status
source-intercom generally_available
source-linkedin-ads generally_available
source-stripe generally_available
source-mixpanel beta
source-notion beta
source-salesloft alpha
source-tempo alpha
source-marketo alpha
source-jira alpha
source-zendesk-chat alpha
source-trello alpha
source-onesignal alpha
source-microsoft-teams alpha
source-twilio alpha
source-linnworks alpha
source-amazon-seller-partner alpha
source-slack alpha
alafanechere commented 2 years ago

I started this work for GA and Beta connectors: https://github.com/airbytehq/airbyte/pull/14924

alafanechere commented 2 years ago

☝️ Done. https://github.com/airbytehq/airbyte/pull/14924 was merged

alafanechere commented 2 years ago

Updated list of non jdbc source connectors with additionalProperties: false:

Connector Release state Version
source-airtable alpha 0.1.2
source-amazon-sqs alpha 0.1.0
source-apify-dataset alpha 0.1.11
source-appstore-singer alpha 0.2.6
source-azure-table alpha 0.1.2
source-chargify alpha 0.1.0
source-chartmogul alpha 0.1.1
source-close-com alpha 0.1.0
source-commercetools alpha 0.1.0
source-confluence alpha 0.1.1
airbyte-customer-io-source alpha 0.1.23
source-delighted alpha 0.1.4
source-dixa alpha 0.1.3
source-dockerhub alpha 0.1.0
source-e2e-test alpha 2.1.1
source-freshsales alpha 0.1.2
source-freshservice alpha 0.1.1
source-gitlab alpha 0.1.6
source-google-workspace-admin-reports alpha 0.1.8
airbyte-harness-source alpha 0.1.23
source-hellobaton alpha 0.1.0
airbyte-jenkins-source alpha 0.1.23
source-kafka alpha 0.2.0
source-kyriba alpha 0.1.0
source-lemlist alpha 0.1.0
source-linnworks alpha 0.1.5
source-looker alpha 0.2.7
source-my-hours alpha 0.1.1
source-onesignal alpha 0.1.2
source-orbit alpha 0.1.1
source-outreach alpha 0.1.2
airbyte-pagerduty-source alpha 0.1.23
source-paystack alpha 0.1.1
source-persistiq alpha 0.1.0
source-pivotal-tracker alpha 0.1.0
source-pokeapi alpha 0.1.5
source-prestashop alpha 0.1.0
source-quickbooks-singer alpha 0.1.5
source-recurly alpha 0.4.1
source-rki-covid alpha 0.1.1
source-salesloft alpha 0.1.3
source-strava alpha 0.1.2
source-talkdesk-explore alpha 0.1.0
source-timely alpha 0.1.0
source-us-census alpha 0.1.2
source-webflow alpha 0.1.2
source-woocommerce alpha 0.1.2
source-zenefits alpha 0.1.0
source-zoom-singer alpha 0.2.4
source-zoho-crm alpha 0.1.0
source-firebolt alpha 0.1.0
source-elasticsearch alpha 0.1.0
alafanechere commented 2 years ago

I'm closing because SAT will not allow uprade made to these connectors if their spec remains with additionalProperties: false. So this issue will eventually be fixed when alpha connectors get an update or enter the certification program.