Jupyter magic doesn't allow inputs to shell commands

[rpwagner]

@zonca I wrote a script to copy the datasets which uses a similar login process as the Globus CLI. And then I learned that Jupyter magic doesn't allow for input to shell commands. So commands like !globus login --no-local-server won't work because the user can't enter in the auth code.

I think we have two solutions:

• Take any script like the copy one which requires Globus tokens and make it an importable Python module. The module can handle login flow when necessary.
• Use the terminal for running scripts.

My preference is for making the scripts importable module files. What do you think?

(Anything we were using the Globus CLI for I can write a Python module for.)

[zonca]

@rpwagner it would be nice to replicate what globus login does using Python. So authenticate and persist credentials somehow. So our current scripts would work fine.

[rpwagner]

@zonca we can do that. The copy script already saves the tokens, it just needs to be updated to make it callable as a Python module. So the user interaction would look like this:

import cheapandfair
cheapandfair.login()
cheapandfair.copydataset('synch', '85017645-30ef-4519-abbb-a73811b914b7:/datasets/synch/')

[rpwagner]

Does that look reasonable?

[zonca]

it would be better if we had the same authentication mechanism that globus-cli uses, so that people login once and then can either use globus-cli or our scripts without reauthenticating. However if this is not possible your solution is good.

[rpwagner]

It does use the same mechanism as the Globus CLI so they won't need to reauthenicate each time, provided we ask for all the correct scopes the first time.

The issue is this they can't authenticate to the Globus CLI in a Jupyter notebook or to a Python CLI script that we write. And the token storage for our Python scripts and the Globus CLI are separate so they would have to authenticate separately to each tool.

This way, we can just use Python modules and they only need to authenticate once. And I think the copy dataset step is the only time they'll need to call Globus, since the file information will be in the manifests.

[zonca]

ok, good

[rpwagner]

I've got a basic version working. It will also provide the user with the hostname of their guest collection, so they just need their UUID and path. I'll update the notebook, too, and get this committed later tonight.

You can see that the user only has to authenticate the first time.

[zonca]

@rpwagner good, then I'll test it and update the rest of the notebooks and scripts if needed

[rpwagner]

@zonca I've update the Python module and the 2_globus.ipynb notebook in #1. Take a look and see if there are any other scripts that will need to call out to Globus or if just the manifest files are enough.