Add the ability to use local variables to store hot/coldkey passwords

roman-opentensor commented 1 week ago

Variable for storing password: (BT_PW_%WALLET_NAME%_%COLDKEY/HOTKEY%_{}, keyname)

Functional:

[x] Function for creating local env variable based on uniq keyfile path;
[x] Encryption/decryption password;
[x] Reading env vars will be applied for encrypted keyfiles only;
[x] Saving env vars will be applied to creating/re-criation keys with passwords;
[x] User will be able to see the logging messages after storing/reading password;
[x] Implement function for storing encrypted password for wallet->key;
[x] Implement function for deleting encrypted password from env ;
[x] add documentation;

AC:

[x] manual tests passed
[x] python tests for bittensor-wallet passed
[x] python tests for bittensor-cli passed
[x] python tests for bittensor-sdk passed

roman-opentensor commented 1 week ago

Brief :)

In terms of manual use, the current implementation has not changed.

The implementation contains the following updates

All keys have a unique identifier for creating a variable in the local environment
- wallet.coldkey_file.env_var_name, wallet.hotkey_file.env_var_name;
The wallet's Keyfile get 2 new methods for saving and deleting an encrypted password to/from a local environment variable
- wallet.coldkey_file.save_password_to_env(password: str) saves passed password or asking type it.
- wallet.hotkey_file.save_password_to_env(password: str)
- wallet.coldkey_file.delete_password_from_env() deletes associated password from local env variable.
- wallet.hotkey_file.delete_password_from_env()
Only encrypted passwords are saved in the local environment variable.
- Before saving the password, it is encrypted. After reading, it is decrypted inside the wallet.
All encryption/decryption methods use the password saved in the local environment variable.
- The process of checking for the presence of an encrypted password in an associative local environment variable applies to all key methods that encrypt and decrypt key data.

Property `env_var_name`

All keys in the wallet have a property env_var_name. This value is unique within your operating system. Below is an example of how you can get this value.

Example: If the key is located at

/Users/example/.bittensor/wallets/hotkeys/default

then name of the local environment variable will be

BT_PW__USERS_EXAMPLE__BITTENSOR_WALLETS_HOTKEYS_DEFAULT

Example getting env_var_name:

from bittensor_wallet import Wallet

my_wallet = Wallet()
my_wallet.hotkey_file.env_var_name
Out[4]: 'BT_PW__USERS_ROMAN__BITTENSOR_WALLETS_DEFAULT_HOTKEYS_DEFAULT'
my_wallet.coldkey_file.env_var_name
Out[5]: 'BT_PW__USERS_ROMAN__BITTENSOR_WALLETS_DEFAULT_COLDKEY'

Encryption/decryption usage

The saved password is used in the following cases:

Unlocking encrypted cold/hot keys
- wallet.unlock_coldkey, wallet.unlock_hotkey methods;
Creating a wallet
- wallet.create method;
recreating a wallet
- wallet.recreate method;
regenerating a hot/cold key
- wallet.regenerate_coldkey, wallet.regenerate_hotkey methods;
creating a keys separately
- wallet.create_new_coldkey, wallet.create_new_hotkey methods;
encrypting and decrypting data for a specific key
- wallet.coldkey_file.encrypt(), wallet.hotkey_file.encrypt(), wallet.coldkey_file.decrypt(), wallet.hotkey_file.decrypt() methods.

Arguments accepted by the `Wallet.create` method:

"""
Checks for existing coldkeypub and hotkeys, and creates them if non-existent.

    Arguments:
        coldkey_use_password (bool): Whether to use a password for coldkey. Defaults to ``True``.
        hotkey_use_password (bool): Whether to use a password for hotkey. Defaults to ``False``.
        save_coldkey_to_env (bool): Whether to save a coldkey password to local env. Defaults to ``False``.
        save_hotkey_to_env (bool): Whether to save a hotkey password to local env. Defaults to ``False``.
        coldkey_password (Optional[str]): Coldkey password for encryption. Defaults to ``None``. If `coldkey_password` is passed, then `coldkey_use_password` is automatically ``True``.
        hotkey_password (Optional[str]): Hotkey password for encryption. Defaults to ``None``. If `hotkey_password` is passed, then `hotkey_use_password` is automatically ``True``.
        overwrite (bool): Whether to overwrite an existing keys. Defaults to ``False``.
        suppress (bool): If ``True``, suppresses the display of the keys mnemonic message. Defaults to ``False``.

    Returns:
        Wallet instance with created keys.
"""

Example of creating a wallet and saving passwords to a local environment variable:

from bittensor_wallet import Wallet
from bittensor_wallet.keyfile import  keyfile_data_is_encrypted

w = Wallet(name="new_wallet", hotkey="myhotkey")
ck_pass = "c_testing"
hk_pass = "h_testing"

w.create(save_coldkey_to_env=True, save_hotkey_to_env=True, coldkey_password=ck_pass, hotkey_password=hk_pass, overwrite=True, suppress=True)

# coldkey file data is encrypted
print(">>> coldkey file data is encrypted", keyfile_data_is_encrypted(w.coldkey_file.data))
# hotkey file data is encrypted
print(">>> hotkey file data is encrypted", keyfile_data_is_encrypted(w.hotkey_file.data))

w.unlock_coldkey()

Result of execution:

Encrypting...
The password has been saved to environment variable 'BT_PW__USERS_ROMAN__BITTENSOR_WALLETS_NEW_WALLET_COLDKEY'.
Encrypting...
The password has been saved to environment variable 'BT_PW__USERS_ROMAN__BITTENSOR_WALLETS_NEW_WALLET_HOTKEYS_MYHOTKEY'.
>>> coldkey file data is encrypted True
>>> hotkey file data is encrypted True

Decrypting...
<Keypair (address=5EF55E6DfkZec2CDxxxxxxxxxxxxxxxxxxxxxxxxxxh)>

UselessXiaoYao commented 2 days ago

This future is not completely implement yet at release v2.0.2 , right?

roman-opentensor commented 1 day ago

This future is not completely implement yet at release v2.0.2 , right?

This is implemented and included in version 2.0.2. Note that this feature (creating a key-linked environment variable) is not enabled by default. Only if you pass the required flag save_coldkey_to_env=True or save_hotkey_to_env=True respectively. Details in the PR description.) Please describe the issue you encountered.

UselessXiaoYao commented 1 day ago

I used to export the password enviroment variable and the run the btcli command to do transfer. But after update bittensor to 8.0. It's not work. And I read the latest code of btwallet v2.0.2, I find the format of variable is changed, but I changed to new format, the btcli transfer also not work, just print "Decrypting...", and then exit. Below is my shell command: export BT_PW__HOME_UBUNTU__BITTENSOR_WALLETS_MYWALLET_COLDKEY="mypassword"; btcli w transfer --dest ANOTHER_WALLET_COLDKEY --verbose --wallet.name mywallet --amount 1.0 And the echo is : Using the wallet path from config: /home/ubuntu/.bittensor/wallets/ Initiating transfer on network: finney Decrypting... (.venv) ubuntu@c2-large-x86-syd-1:~/workspace/yzp$

roman-opentensor commented 2 hours ago

I used to export the password enviroment variable and the run the btcli command to do transfer. But after update bittensor to 8.0. It's not work. And I read the latest code of btwallet v2.0.2, I find the format of variable is changed, but I changed to new format, the btcli transfer also not work, just print "Decrypting...", and then exit. Below is my shell command: export BT_PW__HOME_UBUNTU__BITTENSOR_WALLETS_MYWALLET_COLDKEY="mypassword"; btcli w transfer --dest ANOTHER_WALLET_COLDKEY --verbose --wallet.name mywallet --amount 1.0 And the echo is : Using the wallet path from config: /home/ubuntu/.bittensor/wallets/ Initiating transfer on network: finney Decrypting... (.venv) ubuntu@c2-large-x86-syd-1:~/workspace/yzp$

Could you please provide more context regarding your environment setup? OS, python version in virtual environment, versions of bittensor, bittensor-cli, bittensor-wallet.

I ask this because I have tested this behavior on my side after your message here and it works correctly for me.

Also, keep in mind, that when you export your password via terminal, you have to export encrypted password. We don't want to store your password anywhere without encryption. For example: mypassword is /-/1$,(:!!.

You can store your password with the following code:

from bittensor_wallet import Wallet

my_wallet = Wallet()

# the same approach you can use `for hotkey_file` if this is encrypted.
boolean_result = my_wallet.coldkey_file.save_password_to_env("mypassword")

In bittensor-wallet version 2.0.2 method save_password_to_env returns True in successful case and False if something wrong. I gonna replace the return result to encrypted password. Then you can get it like this:

from bittensor_wallet import Wallet

my_wallet = Wallet()

my_encrypted_password = my_wallet.coldkey_file.save_password_to_env("mypassword")

This may seem like a bit of a complicated process. But we have a responsibility to protect user data, especially passwords.

opentensor / btwallet