rockstor / rockstor-core

Linux/BTRFS based Network Attached Storage(NAS)
http://rockstor.com/docs/contribute_section.html
GNU General Public License v3.0
557 stars 138 forks source link

Adopt dedicated secrets management library #2728

Closed phillxnet closed 11 months ago

phillxnet commented 1 year ago

We have an ongoing need to centralise our secrets management. This need has more recently been highlighted within the following PR/Issue: "Update django-oauth-toolkit #2710" #2727 where we had need to establish a new means to maintain an install-persistent OAUTH_INTERNAL_APP client secret. As this function was deemed to be outside the scope of the linked issue/PR the proposal is being made in this issue. The temporary work-around in #2727 was to establish a per-boot/rockstor-bootstrap.service persistent secret. This however is inappropriate for such facilities as replication where we require install persistence for secrets.

Popular candidates would be:

Proposed initial focus would be our settings.py file.

phillxnet commented 1 year ago

An alternative, but a little more complex/heavy-weight approach here, could be to 'standardise' on keyring: https://github.com/jaraco/keyring

The Python keyring library provides an easy way to access the system keyring service from python. It can be used in any application that needs safe password storage.

This library is more naively concerned with desktop associated keystores, i.e. from the README.rst again:

These recommended keyring backends are supported:

However we have also the option of:

Other keyring implementations are available through Third-Party Backends.

Among these indicated "third-party-backends" we have a potential candidate of:

Where from the linked (in PyPi) GitHub homepage of this project we have:

https://github.com/frispete/keyrings.cryptfile#notes

You can avoid the interactive getpass() request for the keyring password by supplying kr.keyring_key = "your keyring password" before calling any other methods on the keyring. The following example shows a simple way to retrieve the password from an environment variable KEYRING_CRYPTFILE_PASSWORD, when present:

So all a little tricky as we ultimately have to have a de-crypt key (password) for any encryption we rely-upon such as the encrypted local file storage, but I like the idea that we could, without only superficial extensions, switch out the secrets backend used by keyring.

Maybe a half way step here is to at lease adopt the maturity of keyring, to gain our standard interface, and initially side step the tricky encryption requirement for interaction, i.e. entry of a decrypt secret (our host OS has no appropriate user logged in) and initially go with say: (again from https://github.com/jaraco/keyring#third-party-backends):

  • keyrings.alt - "alternate", possibly-insecure backends, originally part of the core package, but available for opt-in.

PyPi page: https://pypi.org/project/keyrings.alt/ Homepage: https://github.com/jaraco/keyrings.alt

But:

Keyrings in this package may have security risks or other implications. These backends were extracted from the main keyring project to make them available for those who wish to employ them, but are discouraged for general production use. Include this module and use its backends at your own risk.

However this is the exact same scenario we face with the use of say python-dotenv: i.e. plain text passwords/secrets !!

But we can iterate to more advanced back-ends as we progress. But this route does seem like a lot of dependencies to move plain-text secrets from one file to another !! Suggestions welcome here. We are, after all, sitting on a base OS with all required options available: but we ultimately still need to open a keystore of some sorts automatically give we have no OS login as-such.

Hooverdan96 commented 1 year ago

With all the reading I did this morning, it seems to me that the keyring stuff is all good especially for centralizing logins/passphrases, etc., however, since it does require the using of some (master) password at some point AND we don't have (as you pointed out) a login process during startup, this would still result in some plaintext password somewhere (or decryptable password by looking at the coding). If that observation holds true (and of course I might have missed something), then I would vote for the (albeit "heavier") route of using the approach suggested for headless Linux system:

https://github.com/jaraco/keyring#using-keyring-on-headless-linux-systems

I assume the Gnome backend will continue to live on for a long time with continuous security improvements, so even if we eventually figure out a process/configuration that allows for "password entry required" upon startup somehow (or read a dongle, HW config, or some quantum physics related entity :)), then the backend piece will be up-to-date and secure.

But, of course all that I have concluded from above might be incorrect ...

phillxnet commented 1 year ago

@Hooverdan96 Thanks, yes this is all quite the challenge.

When that command is started, enter a password into stdin and press Ctrl+D (end of data).

Hence me thinking the crypt file backend would be lighter, and have the same capability, and the same complexity re feeding a password from presumably an env we setup via Poetry or something.

The gnome keyring may be a better bet, given it's native support in the keyring module, than the third party module indicated above. But it may also have more weight/surface area.

Tricky.

Hooverdan96 commented 1 year ago

you are probably right. I think (if I saw this correctly for a test install, disregarding any version pinning that might be required, or other dependencies/already existing items) for a few of the mentioned backends, the sizes are not small:

gnome-keyring - 2.4 MiB. With dependencies (330 packages) installed 513.6 MiB pam_kwallet - 49.2 KiB. With dependencies (196 packages) installed 234.3 MiB keepassxc - 24.2 MiB. With dependencies (61 packages) installed 99.2 MiB

Installing the keyrings cryptfile 1.3-9 with dependencies, seems to be ~17MiB: Size of Dependencies cryptography 41.0.5: 15196.37 KB keyring 24.2.0: 204.22 KB zipp 3.17.0: 35.548 KB pycparser 2.21: 1223.847 KB cffi 1.16.0: 833.995 KB jeepney 0.8.0: 369.157 KB

so, in either case, not totally trivial if you want to keep the footprint relatively small...

phillxnet commented 1 year ago

@Hooverdan96 Thanks, I was wondering about the actual sizes. I've used the gnome-keyring before, but that was on a desktop so no extra was required (gnome back then).

Re the base:

Installing the keyrings

That's nice and small, plus we already have cffi and pycparser and cryptography I think (some incoming via the Django-oath-toolkit I think). poetry show --tree in our /opt/rockstor is good to see our current dependencies re the .venv.

phillxnet commented 11 months ago

A Poetry update introduces more flexibility re env variables: noting here in case this helps us out re establishing install persistent secrets that are, if not found, randomly generated and established via Poetry.

phillxnet commented 11 months ago

We likely have at least 2 regression following on from "Update django-oauth-toolkit https://github.com/rockstor/rockstor-core/issues/2710" https://github.com/rockstor/rockstor-core/pull/2727

phillxnet commented 11 months ago

Regarding a follow-up on:

... feeding a password from presumably an env we setup via Poetry or something.

We are now, post: https://github.com/rockstor/rockstor-core/pull/2755 using the latest Poetry: however without an unofficial plugin (risky for such a critical part of our setup) Poetry does not support environmental variables natively, and it is definitely not favoured as in-scope by the main developers of Poetry:

https://github.com/python-poetry/poetry/issues/337

phillxnet commented 11 months ago

PyPI

phillxnet commented 11 months ago

Potential extension of OS dependency re keyrings.cryptfile: Quick start guide in https://pypi.org/project/keyrings.cryptfile/

or use a local venv, but that will depend on a properly working C compiler and some development packages installed (python-devel and openssl-devel at least).

phillxnet commented 11 months ago

With just the (insufficient) addition of:

keyring = "*"

We have the following changes:

lbuildvm:/opt/rockstor # poetry update
Updating dependencies
Resolving dependencies... (6.4s)

Package operations: 7 installs, 2 updates, 0 removals

  • Updating cryptography (41.0.5 -> 41.0.7)
  • Updating idna (3.4 -> 3.6)
  • Installing jeepney (0.8.0)
  • Installing more-itertools (10.1.0)
  • Installing zipp (3.17.0)
  • Installing importlib-metadata (6.8.0)
  • Installing jaraco-classes (3.3.0)
  • Installing secretstorage (3.3.3)
  • Installing keyring (24.3.0)

Writing lock file

With the first two likely incidental.

Adding:

keyrings.cryptfile = "*"

Results in the following error:

lbuildvm:/opt/rockstor # poetry update

The Poetry configuration is invalid:
  - data.dependencies.keyrings must be valid exactly by one definition (0 matches found)
phillxnet commented 11 months ago

From: https://github.com/python-poetry/poetry/issues/6858 We may be looking for: https://python-poetry.org/docs/cli/#self-add

Some context, apparently Poetry itself uses keyring: https://blog.frank-mich.com/python-poetry-1-0-0-private-repo-issue-fix/

So likely the self-add is relevant there.

FroggyFlox commented 11 months ago

Unrelated to this issue, but wanted to note here for reference: Interesting that it pulls jeepney as a dependency. This actually is my current top candidate to replace python-dbus (https://github.com/rockstor/rockstor-core/issues/2744#issuecomment-1810062592).

phillxnet commented 11 months ago

@FroggyFlox Re:

Interesting that it pulls jeepney as a dependency. This actually is my current top candidate to replace python-dbus

That good to know, especially given the semi-maintained nature of python-dbus that was a bit of a shock.

As a follow-up on this issue; I'm now looking into OS keyring as likely the better place to have this resource located anyway; i.e.:

lbuildvm:/opt/rockstor # zypper info python311-keyring
Retrieving repository 'Update repository of openSUSE Backports' metadata ......................................................[done]
Building repository 'Update repository of openSUSE Backports' cache ...........................................................[done]
Retrieving repository 'Update repository with updates from SUSE Linux Enterprise 15' metadata .................................[done]
Building repository 'Update repository with updates from SUSE Linux Enterprise 15' cache ......................................[done]
Retrieving repository 'Main Update Repository' metadata .......................................................................[done]
Building repository 'Main Update Repository' cache ............................................................................[done]
Loading repository data...
Reading installed packages...

Information for package python311-keyring:
------------------------------------------
Repository     : Update repository with updates from SUSE Linux Enterprise 15
Name           : python311-keyring
Version        : 24.2.0-150400.5.3.1
Arch           : noarch
Vendor         : SUSE LLC <https://www.suse.com/>
Installed Size : 299.1 KiB
Installed      : No
Status         : not installed
Source package : python-keyring-24.2.0-150400.5.3.1.src
Upstream URL   : https://github.com/jaraco/keyring
Summary        : System keyring service access from Python
Description    : 
    The Python keyring lib provides a way to access the system keyring service
    from python. It can be used in any application that needs safe password storage.

But still fathoming our options here. We will likely have to have some related changes in rpmbuild and our installer as a result: but we are then not carrying such sensitive packages that we should be depending on our OS to provide, especially give the final hurdle of needing an on filesystem password !!! I.e. we ultimately have to unlock our passwords without a login !!!

phillxnet commented 11 months ago

My current though on establishing the cryptfile password is to set it to a long random during install, via a one-shot systemd service that embeds into /root a secured file with the master pass within it (in plain text). Ideas on how we get around this ultimate requirement: bar having folks type it in on every boot, are welcome.

phillxnet commented 11 months ago

Re keyring keyrings.alt versions from OS:

As above we have python311-keyring: 24.2.0-150400.5.3.1 = fairly up-to-date.

But the package: python3-keyrings.alt is a little old at 4.0.2 ish:

zypper info python3-keyrings.alt
Loading repository data...
Reading installed packages...

Information for package python3-keyrings.alt:
---------------------------------------------
Repository     : Main Repository
Name           : python3-keyrings.alt
Version        : 4.0.2-bp155.2.9
Arch           : noarch
Vendor         : openSUSE
Installed Size : 153.0 KiB
Installed      : No
Status         : not installed
Source package : python-keyrings.alt-4.0.2-bp155.2.9.src
Upstream URL   : https://github.com/jaraco/keyrings.alt
Summary        : Alternate keyring implementations
Description    : 
    Alternate keyring backend implementations for use with the
    keyring package.

With a PyPi version: https://pypi.org/project/keyrings.alt/ at 5.0.0 where we have in upstream changelog: https://github.com/jaraco/keyrings.alt/blob/main/NEWS.rst

https://github.com/jaraco/keyrings.alt/blob/main/NEWS.rst#v412

https://github.com/jaraco/keyrings.alt/blob/main/NEWS.rst#v420

phillxnet commented 11 months ago

OS python311-keyring

lbuildvm:/opt/rockstor # zypper in --no-recommends python311-keyring
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 10 NEW packages are going to be installed:
  python311-cffi python311-cryptography python311-importlib-metadata python311-jaraco.classes python311-jeepney python311-keyring python311-more-itertools
  python311-pycparser python311-SecretStorage python311-zipp

10 new packages to install.
Overall download size: 2.1 MiB. Already cached: 0 B. After the operation, additional 8.1 MiB will be used.
Continue? [y/n/v/...? shows all options] (y):

We also have the spin-off repo: https://github.com/jaraco/keyrings.alt from the same maintainer where we find our proposed cryptfile backend option as part of a bundle.

And as referenced before: https://pypi.org/project/keyring/#third-party-backends

keyrings.alt - “alternate”, possibly-insecure backends, originally part of the core package, but available for opt-in.

PyPi: https://pypi.org/project/keyrings.alt/

Using OS keyring package as above, plus poetry dependency:

keyrings.alt = "*"

Results in the same:

lbuildvm:/opt/rockstor # poetry update The Poetry configuration is invalid: data.dependencies.keyrings must be valid exactly by one definition (0 matches found)

phillxnet commented 11 months ago

N.B. keyrings.alt does not contain the keyrings.cryptfile backend.

phillxnet commented 11 months ago

Another way we can introduce the master-unlock password, for what-ever we settle on here re backend keystore, is via:

-EnvironmentFile= https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#EnvironmentFile=

which will expose the contents of a file as environment variables, but they are more secure than "Environment=".

The text file should contain newline-separated variable assignments. Empty lines, lines without an "=" separator, or lines starting with ";" or "#" will be ignored, which may be used for commenting. The file must be encoded with UTF-8. Valid characters are unicode scalar values other than unicode noncharacters, U+0000 NUL, and U+FEFF unicode byte order mark. Control codes other than NUL are allowed.

phillxnet commented 11 months ago

It seems we have no clear way to install via Poetry our initially proposed keyrings.cryptfile, and it is not available to the OS.

Keyring very much looks to be desktop orientated, but I'm still considering other options re the backend as the gnome.keyring is massive and pulls in such things as Mesa !!! and x keyboard stuff.

We have the following 3rd party keystores as hopefully more appropriate for our headless approach and more amenable to Poetry or OS install:

From: https://keyring.readthedocs.io/en/latest/#third-party-backends

phillxnet commented 11 months ago

keyring_pass

This is a pass backend for keyring

lbuildvm:~ # zypper info password-store
Loading repository data...
Reading installed packages...

Information for package password-store:
---------------------------------------
Repository     : Main Repository
Name           : password-store
Version        : 1.7.4-bp155.3.7
Arch           : noarch
Vendor         : openSUSE
Installed Size : 63.5 KiB
Installed      : No
Status         : not installed
Source package : password-store-1.7.4-bp155.3.7.src
Upstream URL   : https://zx2c4.com/projects/password-store/
Summary        : Utility to store, retrieve, generate and synchronize passwords
Description    : 
    With password-store, each password lives inside of a gpg encrypted file whose
    filename is the title of the website or resource that requires the password.
    These encrypted files may be organized into meaningful folder hierarchies,
    copied from computer to computer, and, in general, manipulated using standard
    command line file management utilities.

N.B. password-store itself looks to be authored by Jason A. Donenfeld, the author of WireGuard !!

So the only potentially weak link here is the the interface project of keyring_pass, recent but few releases. But this choice would at least get us going again on this issue and instantiate in rockstor-core the keyring client: where the backend could be up for swap-out. But still - we really don't what to have to switch this out later. But we could if need be. Or end up maintaining keyring_pass, or an fork, which is not necessarily a show-stopper. Keyring_pass does look to have Py3.11 compatibiliy currently, and password-store is the secure side of things anyway so we may have our answer on this relatively simple approach - accessible via python or the shell.

phillxnet commented 11 months ago

The result of adding to pyproject.toml just:

keyring-pass = "*"

results in:

lbuildvm:/opt/rockstor # poetry update
Updating dependencies
Resolving dependencies... (5.2s)

Package operations: 8 installs, 0 updates, 0 removals

  • Installing jeepney (0.8.0)
  • Installing more-itertools (10.1.0)
  • Installing zipp (3.17.0)
  • Installing importlib-metadata (6.8.0)
  • Installing jaraco-classes (3.3.0)
  • Installing secretstorage (3.3.3)
  • Installing keyring (23.13.1)
  • Installing keyring-pass (0.8.1)

Writing lock file

Also pulls in keyring, althought it is not the latest at Dec 2022 !! But the secretstorage is latest, as is jaroco-classes and (@FroggyFlox ) jeepney dependency is also latest.

Dependenncies via poetry:


poetry show --tree
...
keyring-pass 0.8.1 https://www.passwordstore.org/ backend for https://pypi.org/project/keyring/
├── jaraco-classes >=3.2.3,<4.0.0
│   └── more-itertools * 
└── keyring >=23.9.3,<24.0.0
    ├── importlib-metadata >=4.11.4 
    │   └── zipp >=0.5 
    ├── jaraco-classes * 
    │   └── more-itertools * 
    ├── jeepney >=0.4.2 
    ├── pywin32-ctypes >=0.2.0 
    └── secretstorage >=3.2 
        ├── cryptography >=2.0 
        │   └── cffi >=1.12 
        │       └── pycparser * 
        └── jeepney >=0.6 (circular dependency aborted here)
phillxnet commented 11 months ago

'password-store' from OS package

as the final backend

lbuildvm:/opt/rockstor # zypper in --no-recommends password-store
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following 4 NEW packages are going to be installed:
  libqrencode4 password-store qrencode tree

4 new packages to install.
Overall download size: 173.0 KiB. Already cached: 0 B. After the operation, additional 353.2 KiB will be used.
Continue? [y/n/v/...? shows all options] (y):

Where removing --no-recommends results in the same package dependencies.

phillxnet commented 11 months ago

As password-store relies on GPG we need to generate a public/private GPG key pair for use in encryption/decryption of our password-store.

Manual: https://www.gnupg.org/documentation/manuals/gnupg/ Unattended use: https://www.gnupg.org/documentation/manuals/gnupg/Unattended-Usage-of-GPG.html#Unattended-Usage-of-GPG

gpg --gen-key can be scripted via for example --batch etc but from: we have: https://www.gnupg.org/documentation/manuals/gnupg/Unattended-GPG-key-generation.html

This is the most flexible way of generating keys, but it is also the most complex one. Consider using the quick key manipulation interface described in the previous subsection “The quick key manipulation interface”.

But from: https://www.gnupg.org/documentation/manuals/gnupg/The-quick-key-manipulation-interface.html we have:

This interface was added mainly for the benefit of GPGME (please consider using GPGME, see the manual subsection “Programmatic use of GnuPG”).

And finally from: https://www.gnupg.org/documentation/manuals/gnupg/Programmatic-use-of-GnuPG.html

Please consider using GPGME instead of calling gpg directly. GPGME offers a stable, backend-independent interface for many cryptographic operations. It supports OpenPGP and S/MIME, and also allows interaction with various GnuPG components.

GPGME provides a C-API, and comes with bindings for C++, Qt, and Python. Bindings for other languages are available.

zypper se -s gpgme
Loading repository data...
Reading installed packages...

S | Name              | Type    | Version            | Arch   | Repository
--+-------------------+---------+--------------------+--------+----------------
  | gpgme             | package | 1.16.0-150400.1.80 | x86_64 | Main Repository
...
i | libgpgme11        | package | 1.16.0-150400.1.80 | x86_64 | Main Repository
...
  | python3-gpgme     | package | 0.3-2.38           | x86_64 | Main Repository

However from:

zypper info python3-gpgme
Loading repository data...
Reading installed packages...

Information for package python3-gpgme:
--------------------------------------
Repository     : Main Repository
Name           : python3-gpgme
Version        : 0.3-2.38
Arch           : x86_64
Vendor         : SUSE LLC <https://www.suse.com/>
Installed Size : 186.0 KiB
Installed      : No
Status         : not installed
Source package : python-gpgme-0.3-2.38.src
Upstream URL   : http://pypi.python.org/pypi/pygpgme
Summary        : A Python module for working with OpenPGP messages
Description    : 
    PyGPGME is a Python module that lets you sign, verify, encrypt and
    decrypt messages using the OpenPGP format.

    It is built on top of the GNU Privacy Guard and the GPGME library.

We have a very old and likely inappropriate package: https://pypi.python.org/pypi/pygpgme Released: Mar 8, 2012

phillxnet commented 11 months ago

GPGME stands for GnuPG made easy:

https://www.gnupg.org/documentation/manuals/gpgme/Generating-Keys.html so really orientated towards programmatic interface only.

So falling back to using the GPG interface created to GPGME:

https://www.gnupg.org/documentation/manuals/gnupg/OpenPGP-Key-Management.html#OpenPGP-Key-Management

gpg --quick-generate-key --yes 0

To generate gpg keys for the root user (UID 0) - asks for password at terminal so we can't script as is.

But if no/empty pass, and we rely on default filesystem security we can do:

lbuildvm:~ # gpg --quick-generate-key --yes --batch --passphrase '' 0
gpg: key 8EF5F61DCF70701D marked as ultimately trusted
gpg: directory '/root/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/root/.gnupg/openpgp-revocs.d/14885BAE6BB08A13CD081B468EF5F61DCF70701D.rev'

And our root user (GID 0) then has:

lbuildvm:~ # gpg --list-secret-keys
/root/.gnupg/pubring.kbx
------------------------
sec   rsa3072 2023-11-29 [SC] [expires: 2025-11-28]
      14885BAE6BB08A13CD081B468EF5F61DCF70701D
uid           [ultimate] 0
ssb   rsa3072 2023-11-29 [E]
phillxnet commented 11 months ago

As we use the --yes option above we do not have idempotency !!

lbuildvm:~ # gpg --quick-generate-key --yes --batch --passphrase '' 0
gpg: A key for "0" already exists
gpg: creating anyway
gpg: key 33EFCFDD42225977 marked as ultimately trusted
gpg: revocation certificate stored as '/root/.gnupg/openpgp-revocs.d/6CFB4EB1D75BFEB93AA65F3233EFCFDD42225977.rev'

and end up rewriting our pgp info:

lbuildvm:~ # gpg --list-secret-keys
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: next trustdb check due at 2025-11-28
/root/.gnupg/pubring.kbx
------------------------
sec   rsa3072 2023-11-29 [SC] [expires: 2025-11-28]
      14885BAE6BB08A13CD081B468EF5F61DCF70701D
uid           [ultimate] 0
ssb   rsa3072 2023-11-29 [E]

sec   rsa3072 2023-11-29 [SC] [expires: 2025-11-28]
      6CFB4EB1D75BFEB93AA65F3233EFCFDD42225977
uid           [ultimate] 0
ssb   rsa3072 2023-11-29 [E]

Resetting via:

rm -rf ~/.gnupg/
...
gpg --list-secret-keys
gpg: directory '/root/.gnupg' created
gpg: keybox '/root/.gnupg/pubring.kbx' created
gpg: /root/.gnupg/trustdb.gpg: trustdb created
...
gpg --list-secret-keys

lists no secrets.

phillxnet commented 11 months ago

Proposed initial creation command (from systemd) via idempotent command:

POC

rm -rf ~/.gnupg/
gpg --quick-generate-key --batch --passphrase '' 0
gpg: directory '/root/.gnupg' created
gpg: keybox '/root/.gnupg/pubring.kbx' created
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key 0F62DA0A0B967F04 marked as ultimately trusted
gpg: directory '/root/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/root/.gnupg/openpgp-revocs.d/0EC0513BD210D1C36E2B93850F62DA0A0B967F04.rev'

echo $?
0

Status:

lbuildvm:~ # gpg --list-secret-keys
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2025-11-28
/root/.gnupg/pubring.kbx
------------------------
sec   rsa3072 2023-11-29 [SC] [expires: 2025-11-28]
      0EC0513BD210D1C36E2B93850F62DA0A0B967F04
uid           [ultimate] 0
ssb   rsa3072 2023-11-29 [E]

Re-runnning the same setup command again:

gpg --quick-generate-key --batch --passphrase '' 0
gpg: A key for "0" already exists

echo $?
2
...
lbuildvm:~ # gpg --list-secret-keys
/root/.gnupg/pubring.kbx
------------------------
sec   rsa3072 2023-11-29 [SC] [expires: 2025-11-28]
      0EC0513BD210D1C36E2B93850F62DA0A0B967F04
uid           [ultimate] 0
ssb   rsa3072 2023-11-29 [E]
phillxnet commented 11 months ago

N.B. the user-id is not a UID as indicated above - but an identifier for the key holder !!!

lbuildvm:~ # gpg --quick-generate-key --batch --passphrase '' rockstor@localhost
gpg: key FFC514297625C5E5 marked as ultimately trusted
gpg: revocation certificate stored as '/root/.gnupg/openpgp-revocs.d/489777C4FEE82AEEBDE4D15DFFC514297625C5E5.rev'
lbuildvm:~ # gpg --list-secret-keys
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: next trustdb check due at 2025-11-28
/root/.gnupg/pubring.kbx
------------------------
sec   rsa3072 2023-11-29 [SC] [expires: 2025-11-28]
      99E3BADB0236B7759384F546E0FE27DF934ED54F
uid           [ultimate] 0
ssb   rsa3072 2023-11-29 [E]

sec   rsa3072 2023-11-29 [SC] [expires: 2025-11-28]
      489777C4FEE82AEEBDE4D15DFFC514297625C5E5
uid           [ultimate] rockstor@localhost
ssb   rsa3072 2023-11-29 [E]
phillxnet commented 11 months ago

Test to see if we have sufficent pgp config in place with our new key/config created.

pass init rockstor@localhost
mkdir: created directory '/root/.password-store/'
Password store initialized for rockstor@localhost

And the creation/addition of a password for our CLIENT_SECRET, via cli could be:

We can also specify the length of the generated password i.e. to replace our SECRET_KEY:

pass generate -n rockstor/SECRET_KEY 100
The generated password for rockstor/SECRET_KEY is:
f1XmtO8Gqxb6SN1oXgkO2VlZAaqc6QFNO7wiCct3vjfh3KwO4XJprFYqmprOBZJ8sxoPeiGhS3ZeLsMN2JbQCkHzTrWGvyvJpoXX

We will likely create via python and the python-keyring, but we may have to script within services or in rpmbuild and the above establishes pass function before moving on to the pass-to-python-keyring shim (keyring-pass) to enable our python-keyring access to password-store managed secrets.

phillxnet commented 11 months ago

Quick .venv test of autoconfig re keyring backend

(rockstor-py3.11) lbuildvm:/opt/rockstor # python
Python 3.11.5 (main, Sep 06 2023, 11:21:05) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import keyring
>>> keyring.
keyring.backend           keyring.core              keyring.delete_password(  keyring.get_credential(   keyring.get_password(     keyring.set_keyring(      keyring.util              
keyring.backends          keyring.credentials       keyring.errors            keyring.get_keyring()     keyring.py312compat       keyring.set_password(     
>>> keyring.get_keyring()
<keyring_pass.PasswordStoreBackend object at 0x7f6cb8d27d90>
phillxnet commented 11 months ago

venv store pass

>>> PASS=secrets.token_urlsafe()
>>> print(PASS)
k9qLgnpKvggVl-toOiysjQY3YM9_Lojh6AvCoTzlTGU
>>> keyring.set_password("rockstor", "TEST", PASS)

cli retrieve:

lbuildvm:~ # pass
Password Store
├── python-keyring
│   └── rockstor
│       └── TEST
└── rockstor
    ├── CLIENT_SECRET
    └── SECRET_KEY

with contents via:

pass python-keyring/rockstor/TEST
k9qLgnpKvggVl-toOiysjQY3YM9_Lojh6AvCoTzlTGU

python venv retrieval of both cli and python created passwords next !!

phillxnet commented 11 months ago

Keyring backed info

keyring has a command line option that we can utilise within our .venv:

(rockstor-py3.11) lbuildvm:/opt/rockstor # which keyring
/opt/rockstor/.venv/bin/keyring
(rockstor-py3.11) lbuildvm:/opt/rockstor # keyring --list-backends
keyring.backends.fail.Keyring (priority: 0)
keyring_pass.PasswordStoreBackend (priority: 1)
keyring.backends.chainer.ChainerBackend (priority: -1)

Help (keyring cli)

(rockstor-py3.11) lbuildvm:/opt/rockstor # keyring --help
usage: keyring [-h] [-p KEYRING_PATH] [-b KEYRING_BACKEND] [--list-backends] [--disable]
               [--print-completion {bash,zsh,tcsh}]
               [{get,set,del}] [service] [username]

positional arguments:
  {get,set,del}
  service
  username

options:
  -h, --help            show this help message and exit
  -p KEYRING_PATH, --keyring-path KEYRING_PATH
                        Path to the keyring backend
  -b KEYRING_BACKEND, --keyring-backend KEYRING_BACKEND
                        Name of the keyring backend
  --list-backends       List keyring backends and exit
  --disable             Disable keyring and exit
  --print-completion {bash,zsh,tcsh}
                        print shell completion script
phillxnet commented 11 months ago

keyring cli set

set

(rockstor-py3.11) lbuildvm:/opt/rockstor # keyring set rockstor CLIENT_SECRET
Password for 'CLIENT_SECRET' in 'rockstor':

get

(rockstor-py3.11) lbuildvm:/opt/rockstor # keyring get rockstor CLIENT_SECRET
jasdoijfaçliejijmçalsdkjfioasejçlajeiajile

pass shell retrieval:

lbuildvm:~ # pass
Password Store
├── python-keyring
│   └── rockstor
│       ├── CLIENT_SECRET
│       └── TEST
└── rockstor
    ├── CLIENT_SECRET
    └── SECRET_KEY
...
lbuildvm:~ # pass python-keyring/rockstor/CLIENT_SECRET
jasdoijfaçliejijmçalsdkjfioasejçlajeiajile
phillxnet commented 11 months ago

Summary

phillxnet commented 11 months ago

set in SHELL via pass

lbuildvm:~ # pass generate -n python-keyring/rockstor/CLIENT_SECRET 100
An entry already exists for python-keyring/rockstor/CLIENT_SECRET. Overwrite it? [y/N] y
The generated password for python-keyring/rockstor/CLIENT_SECRET is:
hmDAsYNQ1R651mpZ7mFH9jxn6WAc2V3TTvYNBXf4wp6CQNvDY7UULjWZHhSPKFfDcVsTplMPv3z94fg4Qmpy0XSUcQfm4S7l4Jah

get_password within .venv via python-keyring

(rockstor-py3.11) lbuildvm:/opt/rockstor # python
Python 3.11.5 (main, Sep 06 2023, 11:21:05) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import keyring
>>> import keyring_pass
>>> keyring.
keyring.backend           keyring.delete_password(  keyring.get_password(     keyring.util
keyring.backends          keyring.errors            keyring.py312compat       
keyring.core              keyring.get_credential(   keyring.set_keyring(      
keyring.credentials       keyring.get_keyring()     keyring.set_password(     
>>> keyring.get_password("rockstor", "CLIENT_SECRET")
'hmDAsYNQ1R651mpZ7mFH9jxn6WAc2V3TTvYNBXf4wp6CQNvDY7UULjWZHhSPKFfDcVsTplMPv3z94fg4Qmpy0XSUcQfm4S7l4Jah'

force generate/refresh password (SHELL):

lbuildvm:~ # pass generate --no-symbols --force python-keyring/rockstor/SECRET_KEY 100
The generated password for python-keyring/rockstor/SECRET_KEY is:
X9zGAt4tQuPUvnd5kR686mmYKVV9IzKw99zyarWMdib1fLoG31MhpgEuiiZoWwKuqtT66IPlTbHMn60qQCrAaOvipOxq1ew7EFIc
lbuildvm:~ # pass generate --no-symbols --force python-keyring/rockstor/SECRET_KEY 100
The generated password for python-keyring/rockstor/SECRET_KEY is:
qP2Fwisv7FGc0g1uv3CRvRiilXibig2RGg1JRqeSANiGbFo9JdR1tGP1Iu43QwaNVNakg8yOU5Av3R32WJk9p9rNopHtk2Wfqb5x

I.e. to cycle our SECRET_KEY predominantly session related Django secret on each update, or boot.

phillxnet commented 11 months ago

Silence output of pass

We need not log/expose the "pass generate ..." password: ergo stdout to /dev/null

lbuildvm:~ # pass generate --no-symbols --force python-keyring/rockstor/SECRET_KEY 100 >/dev/null
lbuildvm:~ #

[EDIT] curiously this was not practicable/workable as-is within systemd ExecStartPre, so for now I've nulled the stdout from rockstor-pre, we log to rockstor.log anyway and stderr is still defaulted to journal.

phillxnet commented 11 months ago

Closing as: Fixed by #2758