Add connector test that validates that the spec uses "valid" JSONSchema

[cgardens]

Problem

For the spec of a connector, we allow the usage of a subset of the functionality that JSONSchema provides. When a spec uses a disabled JSONSchema feature it results in perplexing errors.

Proposed Solution

• We should have a way to validate that a JSONSchema object only uses the features we allow.
• We would it as one of the standard tests for each connectors so that before publishing a connector the developer could bring the spec into compliance.

Put this in OSS Platform and Sources DX backlogs as it will be a quality of life improvement for both.

@sherifnada @evantahler @bleonard for visibility.

[sherifnada]

@cgardens is the allowed subset of json schema described anywhere? I tried documenting as much as i know here over time but I'm not sure if there are other limitations/features.

[cgardens]

So far the subset I have in mind is:

• don't use not, anyOf, or allOf.
• type MUST be set. for pure JSONSchema there are reasons why you might want to leave it unset, but for our use case, doing that is just a headache. See the long conversation Benoit and I had in the linked issue.
• the oneOfs stuff that is mentioned in the docs you linked.
• When airbyte_secret can be used. I think the only types that should have airbyte_secret added to it are string, number, array<string> , and array<number>. But we have to do all of this other bonkers stuff right now to cover our behinds because theoretically someone could naively stick that flag anywhere. A compile time loud failure saves a lot of confusion at runtime.
• Honestly saying that the type field for a spec must either be a value (i.e. type: string) or an array with 1 or 2 values. If 2 values, one of them must be null (i.e. type: [string] or type: [string, null]). Having to reason about type: [object, array, string] is very scary for secrets.
• all of the possible values for a oneOfs should be of the same type. i.e. no oneOf: object, string, array. Again just super scary to deal with clearly especially for secrets. if you're using oneOf, let it be of a oneOf multiple objects and then nest whatever fields you need inside that object.

[sherifnada]

cc @flash1293 related to your interest in additional SATs for the frontend

[flash1293]

Thanks, missed this issue. I copied over all context to https://github.com/airbytehq/airbyte/issues/20063 , closing this to consolidate