We have a method in PyAirbyte called Source.validate_config() but it doesn't run by default. The proposal here is that we enable that at least whenever check is run, if not also before a read() operation.
Initially we left it out of the critical path because of fears it could introduce false-positive failures. What we are seeing a lot more of though, is users receiving confusing errors from the sources themselves.
False positives are still possible, but it's probably a very low-frequency occurance because the platform already requires all JSON Schema validators to pass before the user can save config in hosted versions of Airbyte.
We have a method in PyAirbyte called Source.validate_config() but it doesn't run by default. The proposal here is that we enable that at least whenever
checkis run, if not also before aread()operation.Initially we left it out of the critical path because of fears it could introduce false-positive failures. What we are seeing a lot more of though, is users receiving confusing errors from the sources themselves.
False positives are still possible, but it's probably a very low-frequency occurance because the platform already requires all JSON Schema validators to pass before the user can save config in hosted versions of Airbyte.