Support encryption at rest

[chenqi0805]

As protection to data access in data-prepper, we will support encryption at rest. Encryption requirements TBD.

As a first step, encryption at rest will be applied to db files in service-map.

• LMDBPrepperState: Encryption might be supported written in C. The binding Java API does not expose any encryption API.
• MapDBPrepperState: MapDB 2.0.x used to support XTEA encryption according to https://github.com/jankotek/mapdb/blob/7a7c47f3cb7def1b46cc35747f9a4cf9a8205661/src/main/java/org/mapdb/DBMaker.java#L1092. But this API has been removed in the refactoring of 3.0.x

Note: AWS KMS manages encryption by AES-256: https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/encryption-at-rest.html which is a symmetric cryptographic algorithm(https://en.wikipedia.org/wiki/Advanced_Encryption_Standard). Other cryptographic algorithm is PGP, which uses asymmetric encryption(public-private keys). Although providing more security, it is computationally more expensive.

[chenqi0805]

Alternative solutions:

1. manually encrypt/decrypt the key-value pairs before storing in LMDB/MapDB files.

For manual encryption and decryption, Java has its own JCE framework. The available ciphers are listed as follows:

https://docs.oracle.com/javase/8/docs/api/javax/crypto/Cipher.html https://docs.oracle.com/javase/9/security/java-cryptography-architecture-jca-reference-guide.htm#JSSEC-GUID-2BCFDD85-D533-4E6C-8CE9-29990DEB0190

Issue with manual encryption/decryption:

• Maintain and retrieve encrypted keys
• Maintain secretKeys
• Performance/Efficiency

2. encryption/decryption at serializer/deserializer level: In MapDB, DBMaker allows specifying keySerializer and valueSerializer. One might implement GroupSerializer interface to encode/decode keys/values

[chenqi0805]

Waiting for responses from MapDB and LMDB for plan of support on encryption.

[kowshikn]

@chenqi0805 If you had raised an issue to MapDB and LMDB, please link it here.

[chenqi0805]

Reply from MapDB gitter:

@chenqi0805 yes, I would like to support encryption. It will be done by block encryption (entire store) or via serializers (only part of data such as Map values)
There is new MapDB4 update
https://mapdb.org/blog/mapdb_in_february_2021/

[chenqi0805]

For LMDB, linked issue:

https://github.com/lmdbjava/lmdbjava/issues/171