Enable simple symmetric encryption for password storage on user configuration file

sbrunato commented 5 years ago

Original report by Oyono (Bitbucket: aoyono, GitHub: aoyono).

The way user configuration is done now requires the user to provide its password in clear text in the configuration file. This issue aims at solving this problem by providing a way to crypt this password

sbrunato commented 5 years ago

Original changes by Oyono (Bitbucket: aoyono, GitHub: aoyono).

set milestone to "0.8.0"

sbrunato commented 5 years ago

Original changes by Oyono (Bitbucket: aoyono, GitHub: aoyono).

changed milestone from "0.8.0" to "1.0"

CGuichard commented 2 years ago

State of the art in Eodag

As of now, there exists multiple ways to provide a password/token for each provider :

peps, creodias, onda: <provider>.auth.credentials.password
usgs: <provider>.api.credentials.password
theia: <provider>.auth.credentials.pass
aws_eos, astraea_eod, usgs_satapi_aws, earth_search: <provider>.auth.credentials.aws_secret_access_key
sobloo, mundi: <provider>.auth.credentials.apikey

Each of these fields accept raw value of the credentials. The problem is that if the configuration file is leaked, or shared for debug purposes, the credentials are compromised.

Proposal

Symmetric encryption

As suggested by this issue, it would be better to use a simple symmetric encryption to store the credentials in a more secured way.

The symmetric encryption can be done with cryptography, a widely used Python library.

In this library, a string can be encrypted with a key, and decoded with the same key. Example of key: XKSXP8mwCHN-IE0N5K58rC6VU1_belgYDEb_F-341D0=.

Configuration file

The problem with this issue is how to implement this encryption without adding breaking changes that would break the user's configuration file after upgrading their eodag to a version adding credentials encryption.

We can add a file named "credentials-keys.yml" inside the user's config folder for eodag, alongside of "eodag.yml" and "locations.yml". This file will contain the keys used to encrypt and decrypt the credentials passwords/tokens.

Example of "credentials-keys.yml":

peps: DKqDt8EMTSXpAT4c_uTEnORsOBs1iJG5ssMcID6_7aA=
sobloo: q-HOmuiiNmFCpme7sE6BYZOkmMPHUE4YSfzmoY4vxqw=
aws_eos: i33eNDec267-cNSexsydACFPpyTbzDeWH-F3qmTTi6k=
theia: 2IbOUYku6wqbRNThX-bwqGOwyVWPdHsbbXqbpoL--vE=
usgs: # empty entries are not used

When the config of the providers is loaded from "eodag.yml", if the file "credentials-keys.yml" exists each provider's password/token field inside "eodag.yml" is decrypted if it has a non-empty entry inside "credentials-keys.yml". When the provider isn't inside "credentials-keys.yml" its password/token remains a raw value. In my example, above, the password for peps would be decrypted when loading the config, but the password for creodias would not because it doesn't have an entry inside "credentials-keys.yml". The same can be said for usgs, with an empty entry no decryption is done.

Command

I have described the behavior of eaodag when reading the config with encryption/decryption support. How does the user use it ? I suggest we add a command to manage it.

Example:

$ eodag credentials creodias
password: # use of "getpass" module for password input type
... # Log
OK
$ eodag credentials astraea_eod
aws_secret_access_key: # use of "getpass" module for password input type
... # Log
OK

This command take the user input, encrypt it with a newly generated key, save the key to "credentials-keys.yml", and save the encrypted credential to "eodag.yml". Saving to "eodag.yml" isn't trivial, because when saving with the yaml standard module the order of the keys and the comments aren't preserved. I think modifying only the needed field and not mess with the user config file is important, so we may have to create a kind of custom parser to do it, or use ruamel.yaml, a derivative that keep the key order and the comments.

This proposition is the best way I can find to add password/token symmetric encryption without creating breaking changes.

CGuichard commented 2 years ago

After discussing with @sbrunato, we found a satisfying solution:

The credentials fields will be removed from "user_conf_template.yml" to prevent new users from having raw credentials in the config file, but the credentials fields will be added to "providers.yml" to keep them documented.

All credentials will be encrypted and saved inside a file named "credentials.yml". Encryption and decryption of the credentials will be done with cryptography. The config will load first the user's "eodag.yml", and override/add to the credentials the ones stored in "credentials.yml". If credentials are found in "eodag.yml", a warning will be issued to notify the user that he needs to remove them, and use the new subcommand credentials to create safe credentials.

CS-SI / eodag