Open kennytm opened 3 years ago
I need this feature (only base-64 encoded)
Financial enterprise use lighting tools embedded in shell script to import data routinely without any interaction,and the user password stored in configuration file should be encoded by base-64 algorithm 。
In existing structural configuration (JSON, TOML, YAML), a "string" password field can be naturally extended to a dynamic variable password, which can be used to conceal the secret from the configuration.
A variable is a datatype satisfying the following JSON schema:
Variables can be cascaded, allowing us to use the same vocabulary for providing the password directly or the decryption key & nonce. This also allows us to provide some useful feature like
# read $TIDB_PASSWORD_FILE, which contains a file path, which contains the password itself.
# (this is similar to how GCP's credential file operates)
[tidb]
password.file.env = "TIDB_PASSWORD_FILE"
or some useless security theater like 🙃
[tidb]
password.base64.base64.base64.base64.base64.base64 = "Vm0xNFUxSXhWWGhXYTJSWFlURktWRlpyVWtKUFVUMDk="
TOML's dotted key feature makes it particularly easy to spell out these nested structures. Unfortunately this is not possible in YAML so you need to expand it:
target-database:
host: 127.0.0.1
port: 3306
user: root
password:
aes-256-ctr:
data: {base64: zdpUpVhoJoggKw==}
key: {file: /data/secret/lightning-key.bin}
nonce: {base64: XN74TC92g2MBNbzEPpxZUA==}
We prefer providing dotted command line flags like
./dumpling -h 127.0.0.1 -P 3306 \
-u root \
--password.aes-256-ctr.data.base64 'zdpUpVhoJoggKw==' \
--password.aes-256-ctr.key.file '/data/secret/lightning-key.bin' \
--password.aes-256-ctr.nonce.base64 'XN74TC92g2MBNbzEPpxZUA=='
This, however, requires spf13/pflag#187 or spf13/pflag#199 or spf13/pflag#285 (the amount of duplicated PR shows how well maintained the pflag
library is). If these PRs aren't merged or we can't switch to one of the forks, we may need to use a more conventional and ugly API like:
./dumpling -h 127.0.0.1 -P 3306 \
-u root \
--encrypted-password '{"aes-256-ctr":{
"data":{"base64":"zdpUpVhoJoggKw=="},
"key":{"file":"/data/secret/lightning-key.bin"},
"nonce":{"base64":"XN74TC92g2MBNbzEPpxZUA=="}
}}'
There should be a tool to generate the encrypted password (maybe through tidb-lightning-ctl
/ br debug
/ dmctl
/ tidb-ctl
)
$ ./some-password-tool encrypt base64 -f toml -r 'tidb.password'
Enter password: ••••••••••
## Warning: base64 is not an encryption, it cannot protect your password if leaked. Use at your own risk.
[tidb]
password.base64 = "UGFzc3cwcmQhIQ=="
$ ./some-password-tool encrypt base64 --password 'Passw0rd!!' -f yaml -r 'target-database.password'
## Warning: base64 is not an encryption, it cannot protect your password if leaked. Use at your own risk.
target-database:
password:
base64: "UGFzc3cwcmQhIQ=="
$ ./some-password-tool encrypt base64 --password 'Passw0rd!!' -f json -r ''
// Warning: base64 is not an encryption, it cannot protect your password if leaked. Use at your own risk.
{"base64":"UGFzc3cwcmQhIQ=="}
$ ./some-password-tool encrypt aes-256-ctr --password 'Passw0rd!!' --key-file ./lightning-key.bin | tee encrypted.toml
password.aes-256-ctr.data.base64 = "zdpUpVhoJoggKw=="
password.aes-256-ctr.key.file = "/data/secret/lightning-key.bin"
password.aes-256-ctr.nonce.base64 = "XN74TC92g2MBNbzEPpxZUA=="
The possible subcommands are "base64" and "aes-256-ctr".
Argument | Meaning |
---|---|
-f, --format | Output format: TOML (default), YAML, JSON, CLI |
-r, --root | A dot-separated key path of the root. Default to 'password'. |
-p, --password | The password to encrypt. Reads from stdin if empty. |
-k, --key-file | For aes-256-ctr only. A 32-byte file containing the encryption key. |
the same tool should also be able to decrypt the password.
$ ./some-password-tool decrypt -f toml < encrypted.toml
Passw0rd!!
$ echo '{"x":{"base64":"UGFzc3cwcmQhIQ=="}}' | ./some-password-tool decrypt -f json -r x
Passw0rd!!
$ # with aes-256-ctr, losing the key file should make the decryption output garbage
$ echo -n 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' > /data/secret/lightning-key.bin
$ ./some-password-tool decrypt < encrypted.toml
u÷ dµ�+�Z"
This tool can also be created as a static webpage, which should be the most user-friendly and easiest to use. However, there may be some conception issue about whether the password entered on a web interface will be secretly sent to PingCAP (no it won't).
The library will be placed on pingcap/tidb-tools
for now. We may move it intopingcap/tidb
as a sub-Go-module in future decisions.
Design notes
aes-256-cfb
)dumpling
simply uses pflag
so we're not going to allow ./dumpling encrypt-password ...
tiup
so we can't assume tiup ctl tidb encrypt-password ...
is available./tidb-lightning --tidb-password '...'
for both encrypted and plain text password, by reducing the space of plain text password (breaking backward compatibility)
Feature Request
Is your feature request related to a problem? Please describe:
Currently the MySQL password is stored as plain-text in the
config.toml
, which some users feel uncomfortable with.Describe the feature you'd like:
Provide some way to hide the password. Example:
The password will always be decrypted in the Lightning process no matter which algorithm is chosen, since the MySQL protocol demands the original password for authentication.
Describe alternatives you've considered:
Don't do it. Rely on Lightning-in-SQL.
Teachability, Documentation, Adoption, Optimization: