With #82, transformers only get their subset of the configuration. That's because the prefixes are dynamic to support multiple (and multiple instances of the same) transformers.
However, some transformers need properties from other components, e.g. the ConfluentAvroDecodingTransformer (#128) needs the reader.kafka.topic. To support that, the method getMappingFromRetainedGlobalConfigToLocalConfig(globalConfig: Configuration): Map[String, String] should be added to the StreamTransformerFactory trait. The StreamTransformerAbstractFactory should then extract and map the configuration values according to getMappingFromRetainedGlobalConfigToLocalConfig and add the values with the new keys to the config-subset which is passed to factory.apply.
If mapped keys conflict, an error should be thrown
Comments
Currently, StreamReader and StreamWriter get the full configuration. However, this can change to have a unified behaviour for all Reader, Transformer and Writer and to clearly see cross-component dependencies. To do that, #91 is required.
With #82, transformers only get their subset of the configuration. That's because the prefixes are dynamic to support multiple (and multiple instances of the same) transformers. However, some transformers need properties from other components, e.g. the
ConfluentAvroDecodingTransformer(#128) needs thereader.kafka.topic. To support that, the methodgetMappingFromRetainedGlobalConfigToLocalConfig(globalConfig: Configuration): Map[String, String]should be added to theStreamTransformerFactorytrait. TheStreamTransformerAbstractFactoryshould then extract and map the configuration values according togetMappingFromRetainedGlobalConfigToLocalConfigand add the values with the new keys to the config-subset which is passed tofactory.apply.If mapped keys conflict, an error should be thrown
Comments