When parsing SourceWithSchemaExtraction with sdk.Util.ParseConfig, we get an error saying that there's a type mismatch, because the default value is "avro" (a string), and the target value is an int. The reason is that in Config.Validate() we validate the types.
Possible solution:
Remove type validation in Config.Validate() altogether. In the Connector SDK, it's called only when parsing a config, and after that we decode the map into a struct, where the types are validated too.
Use one of mapstructure's hooks (link), so that developers can implement a MarshalText/UnmarshalText method and customize parsing.
Feature description
The problem: Here's an example from a connector middleware struct:
where
schema.Type
is an alias of schema.Type in conduit-commons.When parsing
SourceWithSchemaExtraction
withsdk.Util.ParseConfig
, we get an error saying that there's a type mismatch, because the default value is"avro"
(a string), and the target value is anint
. The reason is that inConfig.Validate()
we validate the types.Possible solution:
Config.Validate()
altogether. In the Connector SDK, it's called only when parsing a config, and after that we decode the map into a struct, where the types are validated too.mapstructure
's hooks (link), so that developers can implement aMarshalText
/UnmarshalText
method and customize parsing.