cdoron
commented
2 years ago @elsalant : how urgent is this issue?
I think that the best way to address this problem is using Vault, and the Fybrik python vault library: https://github.com/fybrik/fybrik/tree/master/python/vault
@cdoron Some Airbyte connectors, like S3, will require that secrets be passed to them. Currently, the only way this could be done is to put the secret keyword and value in the Asset yaml where they will eventually end up in a mounted file (/etc/conf/conf.yaml) This is of course a security hole. Instead, the secrets - keyword as expected by the Airbyte connector and values - either be stored in Vault or in a Secrets asset and the module should then extract these, append them to the other values specified in Asset yaml and pass these as configuration parameters to the airbyte connector.