@cdoron
Some Airbyte connectors, like S3, will require that secrets be passed to them. Currently, the only way this could be done is to put the secret keyword and value in the Asset yaml where they will eventually end up in a mounted file (/etc/conf/conf.yaml) This is of course a security hole. Instead, the secrets - keyword as expected by the Airbyte connector and values - either be stored in Vault or in a Secrets asset and the module should then extract these, append them to the other values specified in Asset yaml and pass these as configuration parameters to the airbyte connector.
@cdoron Some Airbyte connectors, like S3, will require that secrets be passed to them. Currently, the only way this could be done is to put the secret keyword and value in the Asset yaml where they will eventually end up in a mounted file (/etc/conf/conf.yaml) This is of course a security hole. Instead, the secrets - keyword as expected by the Airbyte connector and values - either be stored in Vault or in a Secrets asset and the module should then extract these, append them to the other values specified in Asset yaml and pass these as configuration parameters to the airbyte connector.