Cannot refresh schema on canned connections

[thomas-gerber]

Describe the bug Airbyte fails to fetch an updated schema for canned connections. It works fine if one create a NEW source, link it to the EXISTING destination.

This is likely caused by one of 2 mismatches: our canned source def mentions GitHub 0.3.8 but:

• Airbyte changes it to 0.2.42, likely because of a version matching thing.
• It is possible that the source config we pass is NOT 0.3.8 (and also NOT 0.2.42) - I can tell because the source config screen looks different between the canned one and when you create a new GitHub source

Suggested course of action: standardize on 0.2.42 for now, and redo the source config.

Or, move to airbyte 40+ and redo the init.

To Reproduce Steps to reproduce the behavior:

1. Start Faros from scratch
2. Put valid settings in the EXISTING GitHub source
3. Go to its connection
4. Click on refresh schema

Note that:

• step 2 is important, as Airbyte will refuse to update schema if the source has not been marked as "discoverable", which happens after a check if I am not mistaken
• this behavior happens on the Jira canned source as well; I suspect it happens on ALL canned sources

Expected behavior All streams are displayed.

Screenshots and logs Worker is fine:

2023-03-03 17:10:30 2023-03-04 01:10:30 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/c74afff5-d342-47bc-b2e3-9884381a1ec3/0/logs.log
2023-03-03 17:10:30 2023-03-04 01:10:30 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.39.37-alpha
2023-03-03 17:10:30 2023-03-04 01:10:30 INFO i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/source-github:0.2.42 exists...
2023-03-03 17:10:30 2023-03-04 01:10:30 INFO i.a.c.i.LineGobbler(voidCall):82 - airbyte/source-github:0.2.42 was found locally.
2023-03-03 17:10:30 2023-03-04 01:10:30 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: c74afff5-d342-47bc-b2e3-9884381a1ec3
2023-03-03 17:10:30 2023-03-04 01:10:30 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/c74afff5-d342-47bc-b2e3-9884381a1ec3/0 --log-driver none --name source-github-discover-c74afff5-d342-47bc-b2e3-9884381a1ec3-0-naayk --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=false -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/source-github:0.2.42 -e AIRBYTE_VERSION=0.39.37-alpha -e WORKER_JOB_ID=c74afff5-d342-47bc-b2e3-9884381a1ec3 airbyte/source-github:0.2.42 discover --config source_config.json
2023-03-03 17:10:32 2023-03-04 01:10:32 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...

but server complains:

2023-03-03 16:50:09 2023-03-04 00:50:09 ERROR i.a.s.e.UncaughtExceptionMapper(toResponse):22 - Uncaught exception
2023-03-03 16:50:09 java.lang.IllegalArgumentException: malformed JsonSchema object type, must have one of the following fields: properties, oneOf, allOf, anyOf in {}
2023-03-03 16:50:09     at io.airbyte.commons.json.JsonSchemas.traverseJsonSchemaInternal(JsonSchemas.java:213) ~[io.airbyte-airbyte-commons-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.commons.json.JsonSchemas.traverseJsonSchemaInternal(JsonSchemas.java:194) ~[io.airbyte-airbyte-commons-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.commons.json.JsonSchemas.traverseJsonSchemaInternal(JsonSchemas.java:206) ~[io.airbyte-airbyte-commons-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.commons.json.JsonSchemas.traverseJsonSchema(JsonSchemas.java:107) ~[io.airbyte-airbyte-commons-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.commons.json.JsonSchemas.traverseJsonSchemaWithFilteredCollector(JsonSchemas.java:142) ~[io.airbyte-airbyte-commons-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.commons.json.JsonSchemas.traverseJsonSchemaWithCollector(JsonSchemas.java:125) ~[io.airbyte-airbyte-commons-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.protocol.models.CatalogHelpers.getFullyQualifiedFieldNamesWithTypes(CatalogHelpers.java:224) ~[io.airbyte.airbyte-protocol-protocol-models-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.protocol.models.CatalogHelpers.getStreamDiff(CatalogHelpers.java:320) ~[io.airbyte.airbyte-protocol-protocol-models-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.protocol.models.CatalogHelpers.lambda$getCatalogDiff$7(CatalogHelpers.java:309) ~[io.airbyte.airbyte-protocol-protocol-models-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
2023-03-03 16:50:09     at io.airbyte.protocol.models.CatalogHelpers.getCatalogDiff(CatalogHelpers.java:305) ~[io.airbyte.airbyte-protocol-protocol-models-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.server.handlers.ConnectionsHandler.getDiff(ConnectionsHandler.java:263) ~[io.airbyte-airbyte-server-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.server.handlers.WebBackendConnectionsHandler.webBackendGetConnection(WebBackendConnectionsHandler.java:261) ~[io.airbyte-airbyte-server-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.server.apis.ConfigurationApi.lambda$webBackendGetConnection$88(ConfigurationApi.java:812) ~[io.airbyte-airbyte-server-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.server.apis.ConfigurationApi.execute(ConfigurationApi.java:873) ~[io.airbyte-airbyte-server-0.39.37-alpha.jar:?]
2023-03-03 16:50:09     at io.airbyte.server.apis.ConfigurationApi.webBackendGetConnection(ConfigurationApi.java:812) ~[io.airbyte-airbyte-server-0.39.37-alpha.jar:?]

[thomas-gerber]

@cjwooo FYI