shotover / shotover-proxy

L7 data-layer proxy
https://docs.shotover.io
Apache License 2.0
83 stars 16 forks source link

Protect Transform - Support Key Encryption Key lookups per row / query #1033

Open benbromhead opened 1 year ago

benbromhead commented 1 year ago

Users need some way of using different key encryption keys based on some properties of the row that is to be protected. We need to further research what's the best way to do this.

E.g.

benbromhead commented 1 year ago

Example for templating:

INSERT (customer_id, name, private_data) INTO some table WHERE customer_id = foo, name = John, private_data = kjsdhksdjhkjsdfhkjsdfh.

- protect:
    - key_lookup_fields: [customer_id]
    - template: "{customer_id}-prodkey"

foo-prodkey is then the key encryption key we ask AWS KMS (or equiv) for

conorbros commented 9 months ago
---
sources:
  - Cassandra:
      name: "cassandra"
      listen_addr: "127.0.0.1:9042"
      chain:
        - Protect:
            key_manager:
              Local:
                keys:
                  customer_id-kek: Ht8M1nDO/7fay+cft71M2Xy7j30EnLAsA84hSUMCm1k=
                  customer_id2-kek: Ht8M1nDO/7fay+cft71M2Xy7j30EnLAsA84hSUMCm1k=
                  otherkey-kek: Ht8M1nDO/7fay+cft71M2Xy7j30EnLAsA84hSUMCm1k=
                  otherkey2-kek: Ht8M1nDO/7fay+cft71M2Xy7j30EnLAsA84hSUMCm1k=
            keyspace_table_columns:
              test_protect_keyspace:
                test_table:
                  col1: "customer_id"
                  col2: "otherkey"
                test_table2:
                  col1: "customer_id2"
                  col2: "otherkey2"
        - CassandraSinkSingle:
            remote_address: "127.0.0.1:9043"
            connect_timeout_ms: 3000