The goal is to have a rust module that can encrypt a connector configuration using sops. There's some special considerations for credentials, though. The connector config JSON schemas use an annotation to identify fields that are considered secrets (secret or airbyte_secret), and we need to ensure that these fields are encrypted in the sops document. Fields that aren't marked as secret should be left in plain text.
The complete sops document will ultimately be returned to the UI in an HTTP response, and the user will be able to edit both the unencrypted and encrypted fields. If they edit an encrypted field, it's contents should be completely replaced with the new plain-text value. The UI will then need to pass the endpoint config back to the backend to re-encrypt it.
When a user edits an endpoint config in the UI, they should see some sort of redacted placeholder for the secret fields (e.g. *****), and the unencrypted values for such fields should of course never be sent in any HTTP response.
This is deployed as a stateless cloud run function. At no point in time should we be storing the configuration.
It accepts a configuration schema, and a configuration document. The schema is used to identify locations which should be encrypted through annotations.
It fully sorts configuration documents on their properties to ensure that, for example, a serde_json::Value representation will correctly pass the sops MAC check.
It encrypts locations using the existing _sops practice w/ the encrypted-suffix sops option, under a KMS key we control.
The goal is to have a rust module that can encrypt a connector configuration using
sops
. There's some special considerations for credentials, though. The connector config JSON schemas use an annotation to identify fields that are considered secrets (secret
orairbyte_secret
), and we need to ensure that these fields are encrypted in the sops document. Fields that aren't marked as secret should be left in plain text.The complete sops document will ultimately be returned to the UI in an HTTP response, and the user will be able to edit both the unencrypted and encrypted fields. If they edit an encrypted field, it's contents should be completely replaced with the new plain-text value. The UI will then need to pass the endpoint config back to the backend to re-encrypt it.
When a user edits an endpoint config in the UI, they should see some sort of redacted placeholder for the secret fields (e.g.
*****
), and the unencrypted values for such fields should of course never be sent in any HTTP response.