passing authentication details to downloadcmd non-interactively

[dmd]

We are working on a project where we want to process hundreds of participants from various datasets (see https://github.com/dmd/rapidtide-cloud ). The general workflow for our tooling is, using AWS Batch:

• Docker container starts, mounting a s3 folder as output
• Retrieve one participant's worth of input data to temporary storage
• Run analysis, writing output to s3
• Remove input data

downloadcmd seems to have been designed solely for interactive use - it expects (see #28 and #20) there to be a human to type in a password, or there to be a keyring service. Neither of those are possible in a container.

downloadcmd used to allow a --password command. What is the alternative to run downloadcmd non-interactively? How can I pass authentication details to it in a situation where no interaction with a human is possible?

[gregmagdits]

For now you can set the password in keyring before starting the nda-tools. (https://pypi.org/project/keyring/). For example you can run the following from a python script

import keyring
keyring.set_password("nda-tools", "myNdaUsername", "myNdaPassword")

Once the credentials are saved in keyring, you can run the nda-tools program without it prompting for a password.

We are planning on changing nda-tools to use the same credentials that you use to login to the website. When we make this change we will make sure there is a way to run the program without needing to enter credentials manually

[dmd]

Hmm. Still no good:

$ python -m venv venvs/nda
$ . venvs/nda/bin/activate
$ pip install nda-tools

[ successful install ]

$ pip install secretstorage --upgrade keyrings.alt

[ successful install ]

$ python
Python 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:40:35) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import keyring
>>> USERNAME='ddrucker'
>>> PASSWORD='mypassword'
>>> keyring.set_password('nda-tools', USERNAME, PASSWORD)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ddrucker/venvs/nda/lib/python3.11/site-packages/keyring/core.py", line 61, in set_password
    get_keyring().set_password(service_name, username, password)
  File "/home/ddrucker/venvs/nda/lib/python3.11/site-packages/keyring/backends/chainer.py", line 55, in set_password
    return keyring.set_password(service, username, password)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ddrucker/venvs/nda/lib/python3.11/site-packages/keyring/backends/SecretService.py", line 87, in set_password
    collection = self.get_preferred_collection()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ddrucker/venvs/nda/lib/python3.11/site-packages/keyring/backends/SecretService.py", line 67, in get_preferred_collection
    raise KeyringLocked("Failed to unlock the collection!")
keyring.errors.KeyringLocked: Failed to unlock the collection!
$

[gregmagdits]

This looks like an issue with the setup of keyring. If this is an Ubuntu machine, remember to run apt-get install -y gnome-keyring

If that's not applicable, then you can email ndahelp@mail.nih.gov to open a support ticket. If necessary we can setup a zoom call to troubleshoot.

[dmd]

I did yesterday, it's # 458927. Haven't heard back yet though.

[dmd]

I have this working in a container now using keyrings.alt and

python -c '
import os, sys, keyring
if not all(key in os.environ for key in ["NDADATASET", "NDAUSER", "NDAPASSWORD"]):
    print("NDADATASET, NDAUSER, and NDAPASSWORD must be set in the environment.")
    sys.exit(1)
keyring.set_password("nda-tools",os.environ["NDAUSER"],os.environ["NDAPASSWORD"])'