Add Globus Auth and Globus Transfer support

[lukaszlacinski]

The changes add:

• the 'token' command with the required 'globus' parameter as an OAuth2 token provider to get an OAuth2 tokens from Globus Auth. (The 'token' command can be extended in the future to support 'esgf' provider to get OAuth2 tokens from ESGF IdPs). Globus OAuth2 tokens are used to authenticate to the Globus Transfer API to submit transfer jobs and monitor them.
• support for 'globus:' protocol to direct download files using Globus Transfer API

[AtefBN]

Hi @lukaszlacinski sorry for the lack of interaction here I just came back from a short vacation. I went through the PR and all looks good to me. Is this still in a draft status?

[lukaszlacinski]

@AtefBN Thank you for checking out the PR code. I have been testing the 'token' command and 'globus' protocol for several weeks running synda once a day in the ESGF monitoring system, https://esgf.anl.gov/monitor/#syndadirect. I noticed some minor issues I have to fix (quick fixes), and then I will remove the draft status.

[AtefBN]

Alright. That's some really good news ! I'll be on the lookout for it then and thank you so much for the effort !

[lukaszlacinski]

@AtefBN The pull request is ready for merging. Thanks.

[AtefBN]

Alright ! Thanks @lukaszlacinski I'll be doing the necessary asap. Thank you once again for the effort!

[YanchunHe]

Hi, thanks for your work! But I have a problem with the globus authentication:

$ synda get  CMIP6.CMIP.NCC.NorESM2-MM.historical.r1i1p1f1.Omon.tos.gn.v20191108
5 file(s) will be downloaded for a total size of 161.5 MB.
Globus tokens not found (use 'renew' command to retrieve new tokens).

Alought I have set in:

$ cat ~/.synda/conf/sdt.conf
[module]
download = true
post_processing = false
globustransfer = false

I also tried to get an globus token by:

$ synda token -p globus renew
Native App Authorization URL:
https://auth.globus.org/v2/oauth2/authorize?code_challenge=BrmiBhFVVuHVNyGDj6hn5N8M1-EKJNnNgptobIsbTqI&state=_default&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&response_type=code&client_id=83ec00c1-e67a-4356-9f1f-f7e31177e31a&scope=openid+email+profile+urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall&code_challenge_method=S256&access_type=offline
Enter the auth code:

But I find it is quite complicate to get a "native app" auth code (refer to: https://docs.globus.org/api/transfer/overview/#thick_clients)

Any suggestions?

Thanks in advance!

Yanchun

[YanchunHe]

I find paste the above URL to web browser can get an auth code (with globus signed in) (if fails, try again). Thus fix this problem.