First time using the client side "oauth-ssh-token authorize" command fails

[tscollins-nygc]

I did the following per the documentation:

"# yum install python2-pip.noarch" $ echo "PATH=${PATH}:~/.local/bin" >> ~/.bash_profile $ echo "export PATH" >> ~/.bash_profile $ . ~/.bash_profile $ pip install --user oauth-ssh

but when trying to generate/create/authorize a token:

[tscollins@tscollins-vm ~]$ oauth-ssh-token authorize ssh.tscwork.net No handlers could be found for logger "paramiko.transport" Traceback (most recent call last): File "/nethome/tscollins/.local/bin/oauth-ssh-token", line 11, in sys.exit(oauth_ssh_token()) File "/nethome/tscollins/.local/lib/python2.7/site-packages/click/core.py", line 722, in call return self.main(args, kwargs) File "/nethome/tscollins/.local/lib/python2.7/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/nethome/tscollins/.local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/nethome/tscollins/.local/lib/python2.7/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, ctx.params) File "/nethome/tscollins/.local/lib/python2.7/site-packages/click/core.py", line 535, in invoke return callback(args, *kwargs) File "/nethome/tscollins/.local/lib/python2.7/site-packages/oauth_ssh/oauth_ssh_token.py", line 88, in wrapper func(args, **kw) File "/nethome/tscollins/.local/lib/python2.7/site-packages/oauth_ssh/oauth_ssh_token.py", line 124, in token_authorize policy = SSHService(fqdn, port).get_security_policy() File "/nethome/tscollins/.local/lib/python2.7/site-packages/oauth_ssh/ssh_service.py", line 26, in get_security_policy transport = Transport(self._fqdn, self._port) File "/nethome/tscollins/.local/lib/python2.7/site-packages/oauth_ssh/transport.py", line 120, in init self.start_client(timeout=15) File "/nethome/tscollins/.local/lib/python2.7/site-packages/paramiko/transport.py", line 660, in start_client raise e AttributeError: Raw

and then login of course fails:

[tscollins@tscollins-vm ~]$ oauth-ssh tscollins@ssh.tscwork.net No token found. Use oauth-ssh-token authorize ssh.tscwork.net.

[tscollins-nygc]

Also when trying just plain old ssh:

[tscollins@tscollins-vm ~]$ ssh tscollins@ssh.tscwork.net Password: Enter your OAuth token:

since I do not have a OAuth token I just Ctrl+c

[JasonAlt]

Could you run

python --version pip list --verbose

[tscollins-nygc]

[tscollins@tscollins-vm ~]$ python --version Python 2.7.5 [tscollins@tscollins-vm ~]$ pip list --verbose | more asn1crypto (0.24.0) Babel (0.9.6) backports.ssl-match-hostname (3.5.0.1) bcrypt (3.1.7) certifi (2019.6.16) cffi (1.12.3) chardet (3.0.4) click (6.7) configobj (4.7.2) cryptography (2.4.2) decorator (3.4.0) enum34 (1.1.6) futures (3.1.1) idna (2.8) iniparse (0.4) iotop (0.6) ipaddress (1.0.16) Jinja2 (2.7.2) kitchen (1.1.1) M2Crypto (0.21.1) MarkupSafe (0.11) msgpack-python (0.5.6) oauth-ssh (0.9) paramiko (2.6.0) perf (0.1) pip (8.1.2) pycparser (2.19) pycrypto (2.6.1) pycurl (7.19.0) pygobject (3.22.0) pygpgme (0.3) pyliblzma (0.5.3) PyNaCl (1.3.0) python-linux-procfs (0.4.9) pyudev (0.15) pyxattr (0.5.1) PyYAML (3.10) pyzmq (14.7.0) requests (2.22.0) salt (2015.5.10) schedutils (0.4) setuptools (0.9.8) six (1.9.0) slip (0.4.0) slip.dbus (0.4.0) SSSDConfig (1.16.2) urlgrabber (3.10) urllib3 (1.25.3) yum-metadata-parser (1.1.4)

[tscollins-nygc]

[tscollins@tscollins-vm ~]$ cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core)

[tscollins-nygc]

If I need to downgrade the version of python installed or even the OS I am more then willing to do so but the documentation does say Python 2.7+

[JasonAlt]

That likely won't be necessary. At the time of release, paramiko+cryptography had some unnecessary deprecation warnings. paramiko devs recommended tying to a specific version of cryptography until a later version of paramiko became available. Now that it has, it breaks on authorize. The fix we'll test will use the updated versions of these packages.

I'll have something to test soon.

[tscollins-nygc]

Thanks, looking forward to testing a new bit of code :)

[JasonAlt]

Please give the new release a try:

pip install --user oauth-ssh --upgrade

[tscollins-nygc]

Install of upgrade went OK

[tscollins@tscollins-vm ~]$ pip install --user oauth-ssh --upgrade Collecting oauth-ssh Downloading https://files.pythonhosted.org/packages/cb/55/78fba342091f66d54d09920df04cc883a7747023797e4974a1d948bac92b/oauth_ssh-0.10-py2.py3-none-any.whl Requirement already up-to-date: click<7.0,>=6.7 in ./.local/lib/python2.7/site-packages (from oauth-ssh) Requirement already up-to-date: paramiko==2.6.0 in ./.local/lib/python2.7/site-packages (from oauth-ssh) Requirement already up-to-date: requests<3.0,>=2.21.0 in ./.local/lib/python2.7/site-packages (from oauth-ssh) Collecting cryptography==2.7 (from oauth-ssh) Downloading https://files.pythonhosted.org/packages/e6/68/50698ce24c61db7d44d93a5043c621a0ca7839d4ef9dff913e6ab465fc92/cryptography-2.7-cp27-cp27mu-manylinux1_x86_64.whl (2.3MB) 100% |████████████████████████████████| 2.3MB 520kB/s Requirement already up-to-date: pynacl>=1.0.1 in ./.local/lib/python2.7/site-packages (from paramiko==2.6.0->oauth-ssh) Requirement already up-to-date: bcrypt>=3.1.3 in ./.local/lib/python2.7/site-packages (from paramiko==2.6.0->oauth-ssh) Requirement already up-to-date: chardet<3.1.0,>=3.0.2 in ./.local/lib/python2.7/site-packages (from requests<3.0,>=2.21.0->oauth-ssh) Requirement already up-to-date: idna<2.9,>=2.5 in ./.local/lib/python2.7/site-packages (from requests<3.0,>=2.21.0->oauth-ssh) Requirement already up-to-date: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./.local/lib/python2.7/site-packages (from requests<3.0,>=2.21.0->oauth-ssh) Requirement already up-to-date: certifi>=2017.4.17 in ./.local/lib/python2.7/site-packages (from requests<3.0,>=2.21.0->oauth-ssh) Requirement already up-to-date: enum34; python_version < "3" in ./.local/lib/python2.7/site-packages (from cryptography==2.7->oauth-ssh) Requirement already up-to-date: asn1crypto>=0.21.0 in ./.local/lib/python2.7/site-packages (from cryptography==2.7->oauth-ssh) Requirement already up-to-date: cffi!=1.11.3,>=1.8 in ./.local/lib/python2.7/site-packages (from cryptography==2.7->oauth-ssh) Collecting six>=1.4.1 (from cryptography==2.7->oauth-ssh) Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl Collecting ipaddress; python_version < "3" (from cryptography==2.7->oauth-ssh) Downloading https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl Requirement already up-to-date: pycparser in ./.local/lib/python2.7/site-packages (from cffi!=1.11.3,>=1.8->cryptography==2.7->oauth-ssh) Installing collected packages: six, ipaddress, cryptography, oauth-ssh Found existing installation: cryptography 2.4.2 Uninstalling cryptography-2.4.2: Successfully uninstalled cryptography-2.4.2 Found existing installation: oauth-ssh 0.9 Uninstalling oauth-ssh-0.9: Successfully uninstalled oauth-ssh-0.9 Successfully installed cryptography-2.7 ipaddress-1.0.22 oauth-ssh-0.10 six-1.12.0

[tscollins@tscollins-vm ~]$ oauth-ssh-token authorize ssh.tscwork.net The authenticity of host 'ssh.tscwork.net' can't be established. ED25519 key fingerprint is SHA256:AxxPDwsFjEKTqaEYE21sV6emUyx6NaYiT8UGb1tFN6c= Are you sure you want to continue connecting (yes/no)? yes Authorization to this host has failed. Likely causes are no local account or a misconfigured service.

I can say with certainty that there is a local account for tscollins on ssh.tscwork.net so wondering what logs you need from the client/server system to further debug this issue?

[JasonAlt]

The authorize step connects as special user oauth-ssh in order to query the SSH service for the security policy info that is necessary for the authorization step. That account should have been created during the RPM install. Can you verify that it exists on the server?

[tscollins-nygc]

Verified that user tscollins and oauth-ssh accounts exist on the server:

[root@ssh.tscwork.net Tue Aug 27-10:25 AM ~] id tscollins uid=1000(tscollins) gid=1000(tscollins) groups=1000(tscollins),10(wheel) [root@ssh.tscwork.net Tue Aug 27-10:25 AM ~] id oauth-ssh uid=981(oauth-ssh) gid=975(oauth-ssh) groups=975(oauth-ssh)

and here is the account info on the client system

[tscollins@tscollins-vm.nygenome.org Tue Aug 27-10:27 AM ~]$ id tscollins uid=10046(tscollins) gid=100(users) groups=100(users),90262(starfish_admins),10329(informatics),10003(pipelineops),30285(cust_cgnd),9001(file_xfer),9018(rescomp),9041(gdan),9005(prod),4(adm) [tscollins@tscollins-vm.nygenome.org Tue Aug 27-10:27 AM ~]$ id oauth-ssh id: oauth-ssh: no such user

[JasonAlt]

It looks like something is definitely misconfigured on the server. This is how your service is responding:

[jasonalt@localhost ~]$ ssh -l oauth-ssh ssh.tscwork.net Password:

And this is an example of how the service should be responding:

[jasonalt@localhost ~]$ ssh -l oauth-ssh ssh.demo.globus.org Enter your OAuth token:

The password prompt suggests that the oauth-ssh pam module is not configured to handle the request. Check your sshd.conf settings and pam/sshd settings:

https://github.com/XSEDE/oauth-ssh/tree/master/server#configure-sshd-to-use-pam https://github.com/XSEDE/oauth-ssh/tree/master/server#configure-pam-to-use-oauth-ssh

[tscollins-nygc]

This is how it responds when a valid local username is provide:

[tscollins@tscollins-vm.nygenome.org Tue Aug 27-10:58 AM ~]$ ssh -l oauth-ssh tscollins@ssh.tscwork.net Password: Enter your OAuth token:

it seems that it is using both the password login option and then it seems to be trying the OAuth token. Here is a dump of the servers sshd_config:

HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostKey /etc/ssh/ssh_host_ed25519_key SyslogFacility AUTHPRIV AuthorizedKeysFile .ssh/authorized_keys PasswordAuthentication yes ChallengeResponseAuthentication yes GSSAPIAuthentication yes GSSAPICleanupCredentials no UsePAM yes X11Forwarding yes AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS Subsystem sftp /usr/libexec/openssh/sftp-server

The two values the documentation says to have set (UsePAM and ChallengeResponseAuthentication) are but I am wondering if I should set 'no' for 'PasswordAuthentication'? Also the /etc/pam.d/sshd file has the five entries that are required:

auth required pam_sepermit.so auth required pam_env.so auth [success=done maxtries=die new_authtok_reqd=done default=ignore] pam_oauth_ssh.so auth requisite pam_succeed_if.so uid >= 1000 quiet_success auth required pam_deny.so auth substack password-auth auth include postlogin -auth optional pam_reauthorize.so prepare account required pam_nologin.so account include password-auth password include password-auth session required pam_selinux.so close session required pam_loginuid.so session required pam_selinux.so open env_params session required pam_namespace.so session optional pam_keyinit.so force revoke session include password-auth session include postlogin -session optional pam_reauthorize.so prepare

Should any of the other lines be removed? Maybe 'password-auth'?

[JasonAlt]

Enabling PasswordAuthentication yes on my system didn't cause the service to issue the password prompt; it just issued the token prompt. So I'm at a loss. Might there be something in your client-side config that is preferring password over token?

[tscollins-nygc]

The documentation for the client side doesn't mention making any modifications to /etc/ssh/sshd_config or /etc/pam.d/sshd but here is the default client /etc/ssh/sshd_config

HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostKey /etc/ssh/ssh_host_ed25519_key SyslogFacility AUTHPRIV PermitRootLogin yes AuthorizedKeysFile .ssh/authorized_keys PasswordAuthentication yes ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials no UsePAM yes X11Forwarding yes Banner /etc/issue.net AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS Subsystem sftp /usr/libexec/openssh/sftp-server

Should 'ChallengeResponseAuthentication' be set to 'yes' on the client as well? Also the /etc/pam.d/sshd

auth required pam_sepermit.so auth substack password-auth auth include postlogin -auth optional pam_reauthorize.so prepare account required pam_nologin.so account include password-auth password include password-auth session required pam_selinux.so close session required pam_loginuid.so session required pam_selinux.so open env_params session required pam_namespace.so session optional pam_keyinit.so force revoke session include password-auth session include postlogin -session optional pam_reauthorize.so prepare

does it need these extra lines as well:

auth required pam_env.so auth [success=done maxtries=die new_authtok_reqd=done default=ignore] pam_oauth_ssh.so auth requisite pam_succeed_if.so uid >= 1000 quiet_success auth required pam_deny.so

[JasonAlt]

I was referring to /etc/ssh/ssh_config and (more likely) ~/.ssh/config on the client side. Especially any options like PreferredAuthentications. These are not modified specifically for oauth-ssh, but they may be modified for other hosts.

[tscollins-nygc]

client side:

[tscollins@tscollins-vm.nygenome.org Tue Aug 27-02:58 PM ~]$ grep -v # /etc/ssh/ssh_config Host * GSSAPIAuthentication yes ForwardX11Trusted yes SendEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES SendEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT SendEnv LC_IDENTIFICATION LC_ALL LANGUAGE SendEnv XMODIFIERS

and no config in ~/.ssh/

[tscollins-nygc]

I don't think it matters but the client documentation says to install python-pip which does not exist

[root@tscollins-vm ~]# yum search python-pip Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Warning: No matches found for: python-pip No matches found

There is a package called pyhton2-pip which is what I installed

[root@tscollins-vm ~]# yum search python2-pip Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile python2-pip.noarch : A tool for installing and managing Python 2 packages

and it does come from the EPEL repo

[root@tscollins-vm ~]# yum info python2-pip Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Available Packages Name : python2-pip Arch : noarch Version : 8.1.2 Release : 6.el7 Size : 1.7 M Repo : EPEL Summary : A tool for installing and managing Python 2 packages URL : http://www.pip-installer.org License : MIT Description : Pip is a replacement for easy_install : <http://peak.telecommunity.com/DevCenter/EasyInstall>_. It uses mostly the : same techniques for finding packages, so packages that were made : easy_installable should be pip-installable as well.

I have just rebuilt the client VM and want to be sure I am installing what are the supported packages.

[JasonAlt]

It looks like you may have had it working around Tue 27 Aug 2019 03:57:16 PM CDT:

[jasonalt@localhost tmp]$ ssh -l oauth-ssh ssh.tscwork.net Enter your OAuth token:

Can you report what changed?

[tscollins-nygc]

Went through the server documentation again:

[root@ssh.tscwork.net Tue Aug 27 ~]#/usr/sbin/oauth-ssh-config register ssh.tscwork.net Success [root@ssh.tscwork.net Tue Aug 27 ~]#vi /etc/ssh/sshd_config [root@ssh.tscwork.net Tue Aug 27 ~]#systemctl restart sshd.service

Verified the UsePAM and ChallengeResponseAuthentication are set right in the config and restarted sshd, checked the status

[root@ssh.tscwork.net Tue Aug 27 ~]#systemctl status sshd.service ● sshd.service - OpenSSH server daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2019-08-27 16:50:15 EDT; 5s ago Docs: man:sshd(8) man:sshd_config(5) Main PID: 22609 (sshd) Tasks: 2 CGroup: /system.slice/sshd.service ├─22594 sshd: [accepted] └─22609 /usr/sbin/sshd -D Aug 27 16:50:15 ssh.tscwork.net systemd[1]: Starting OpenSSH server daemon... Aug 27 16:50:15 ssh.tscwork.net sshd[22609]: Server listening on 0.0.0.0 port 22. Aug 27 16:50:15 ssh.tscwork.net sshd[22609]: Server listening on :: port 22. Aug 27 16:50:15 ssh.tscwork.net systemd[1]: Started OpenSSH server daemon.

[root@ssh.tscwork.net Tue Aug 27 ~]#vi /etc/pam.d/sshd

Added the five lines that the documentation states and then rebooted the system to make sure everything was reloaded. Then from the client:

[tscollins@tscollins-vm.nygenome.org Tue Aug 27 ~]$ssh -l oauth-ssh ssh.tscwork.net Enter your OAuth token:

So did a Ctrl+c to break out of it and tried the rest of the client steps

[tscollins@tscollins-vm.nygenome.org Tue Aug 27 ~]$oauth-ssh-token authorize ssh.tscwork.net The authenticity of host 'ssh.tscwork.net' can't be established. ED25519 key fingerprint is SHA256:AxxPDwsFjEKTqaEYE21sV6emUyx6NaYiT8UGb1tFN6c= Are you sure you want to continue connecting (yes/no)? yes Please go to this URL and login: https://auth.globus.org/v2/oauth2/authorize?code_challenge=dSNzWQyz31l25_ip9KaTlCxjCcvBiBRVoIwGtVEKl4c&state=_default&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&code_challenge_method=S256&client_id=f8aa2b77-dafa-471d-af3b-5f5c6129eb82&scope=https%3A%2F%2Fauth.globus.org%2Fscopes%2Fssh.tscwork.net%2Fssh&access_type=offline&response_type=code Please enter the code you get after login here: F9yo5aqhzCUhhz2svIG5sfj1AKChA0 [tscollins@tscollins-vm.nygenome.org Tue Aug 27 ~]$oauth-ssh ssh.tscwork.net Could not determine remote account to use. Please use -l . [tscollins@tscollins-vm.nygenome.org Tue Aug 27 ~]$oauth-ssh -l tscollins ssh.tscwork.net Unexpected reply from the SSH service: [tscollins@tscollins-vm.nygenome.org Tue Aug 27 ~]$oauth-ssh tscollins@ssh.tscwork.net Unexpected reply from the SSH service: [tscollins@tscollins-vm.nygenome.org Tue Aug 27 ~]$ssh tscollins@ssh.tscwork.net Enter your OAuth token: Password: Enter your OAuth token: Password: Enter your OAuth token: Authentication failed.

[tscollins-nygc]

took a quick look through /var/log/secure for my login attempts and saw this line:

Aug 27 17:15:37 ssh sshd[3217]: error: PAM: Authentication failure for tscollins from vpn.nygenome.org

[tscollins-nygc]

@JasonAlt are there any logs you need me to collect from server/client to help you debug this issue?

[JasonAlt]

[tscollins@tscollins-vm.nygenome.org Tue Aug 27 ~]$ssh -l oauth-ssh ssh.tscwork.net Enter your OAuth token:

So did a Ctrl+c to break out of it and tried the rest of the client steps

Do you know what changed to fix it, so I can add it to the documentation? I see you still get password prompts though. Odd because it seems we are using the same ec2 instance type.

There are a couple of things you can try.

1. Enable debugging in the oauth ssh pam module.

   • edit /etc/pam.d/sshd and add debug to the end of the pam_oauth_ssh line. restart sshd
   • edit /etc/rsyslog and choose a place to log debug info. Add *.debug <path_to_file>. Restart rsyslogd.
   • reconnect with oauth-ssh. careful with the debug output, it may contain access tokens and such.

2. Use SSH and provide the access token to see what the returned value is from the service:

   • run: oauth-ssh ssh.tscwork.net This will fail but it'll ensure you have a valid token for the next step.
   • run: oauth-ssh-token show token ssh.tscwork.net This is your access token.
   • run: ssh -l tscollins ssh.tscwork.net and paste in the access token.

[tscollins-nygc]

Made the changes and tried to connect from client:

[tscollins@tscollins-vm.nygenome.org Wed Aug 28 ~]$oauth-ssh ssh.tscwork.net Could not determine remote account to use. Please use -l . [tscollins@tscollins-vm.nygenome.org Wed Aug 28 ~]$oauth-ssh -l tscollins ssh.tscwork.net Unexpected reply from the SSH service: [tscollins@tscollins-vm.nygenome.org Wed Aug 28 ~]$ssh -l tscollins ssh.tscwork.net Enter your OAuth token: Password: Enter your OAuth token:

From the server: cat /var/log/globusauth Aug 28 22:57:03 ssh systemd: Stopping System Logging Service... Aug 28 22:57:03 ssh rsyslogd: [origin software="rsyslogd" swVersion="8.24.0-34.el7" x-pid="2565" x-info="http://www.rsyslog.com"] exiting on signal 15. Aug 28 22:57:03 ssh systemd: Stopped System Logging Service. Aug 28 22:57:03 ssh polkitd[710]: Unregistered Authentication Agent for unix-process:3247:8895855 (system bus name :1.1655, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus) Aug 28 22:57:42 ssh polkitd[710]: Registered Authentication Agent for unix-process:3270:8899737 (system bus name :1.1656 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) Aug 28 22:57:42 ssh systemd: Starting System Logging Service... Aug 28 22:57:42 ssh rsyslogd: [origin software="rsyslogd" swVersion="8.24.0-34.el7" x-pid="3276" x-info="http://www.rsyslog.com"] start Aug 28 22:57:42 ssh polkitd[710]: Unregistered Authentication Agent for unix-process:3270:8899737 (system bus name :1.1656, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus) Aug 28 22:57:42 ssh systemd: Started System Logging Service. Aug 28 22:58:01 ssh systemd: Created slice User Slice of pcp. Aug 28 22:58:01 ssh systemd: Started Session 312 of user pcp. Aug 28 22:58:01 ssh CROND[3292]: (pcp) CMD ( /usr/libexec/pcp/bin/pmie_check -C) Aug 28 22:58:01 ssh systemd: Removed slice User Slice of pcp. Aug 28 22:58:20 ssh sshd[3329]: fatal: input_userauth_info_response: wrong number of replies [preauth] Aug 28 22:58:20 ssh dbus[736]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper) Aug 28 22:58:21 ssh dbus[736]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd' Aug 28 22:58:21 ssh setroubleshoot: SELinux is preventing /usr/sbin/sshd from write access on the directory /etc/pki/nssdb. For complete SELinux messages run: sealert -l 07abde7e-68b1-4c66-98c2-53d42cee25cc Aug 28 22:58:21 ssh python: SELinux is preventing /usr/sbin/sshd from write access on the directory /etc/pki/nssdb.#012#012* Plugin catchall (100. confidence) suggests **#012#012If you believe that sshd should be allowed write access on the nssdb directory by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'sshd' --raw | audit2allow -M my-sshd#012# semodule -i my-sshd.pp#012 Aug 28 22:58:47 ssh dbus[736]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper) Aug 28 22:58:48 ssh dbus[736]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd' Aug 28 22:58:49 ssh setroubleshoot: SELinux is preventing /usr/sbin/sshd from write access on the directory /etc/pki/nssdb. For complete SELinux messages run: sealert -l 07abde7e-68b1-4c66-98c2-53d42cee25cc Aug 28 22:58:49 ssh python: SELinux is preventing /usr/sbin/sshd from write access on the directory /etc/pki/nssdb.#012#012* Plugin catchall (100. confidence) suggests **#012#012If you believe that sshd should be allowed write access on the nssdb directory by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'sshd' --raw | audit2allow -M my-sshd#012# semodule -i my-sshd.pp#012 Aug 28 22:58:52 ssh sshd[3351]: error: PAM: Authentication failure for tscollins from vpn.nygenome.org Aug 28 22:58:53 ssh sshd[3351]: Connection closed by 69.80.224.20 port 43826 [preauth] Aug 28 22:59:27 ssh polkitd[710]: Registered Authentication Agent for unix-process:3383:8910197 (system bus name :1.1663 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) [root@ssh ~]#

[JasonAlt]

Two things stand out in that log:

1. selinux is preventing sshd from doing something in /etc/pki/nssdb. I don't believe this is related but we can test it by disabling selinux sudo setenforce 0 and trying oauth-ssh again.

2. And this error condition with I can not duplicate:

   fatal: input_userauth_info_response: wrong number of replies

Can you retry oauth-ssh with debug enabled to see if it logs this again to be sure this is related?

[tscollins-nygc]

@JasonAlt I built a new server instance in Google Compute, FQDN is ga-test.tscwork.net, and it seems to work now: tscollins@tscollins-vm:~$ ssh -l oauth-ssh ga-test.tscwork.net Enter your OAuth token:

130 tscollins@tscollins-vm:~$ oauth-ssh-token authorize ga-test.tscwork.net The authenticity of host 'ga-test.tscwork.net' can't be established. ED25519 key fingerprint is SHA256:UlgGWFQMh2g0kpg/GaCAQ+7LYWSgcwRhs7QN7BYCpX4= Are you sure you want to continue connecting (yes/no)? yes Please go to this URL and login: https://auth.globus.org/v2/oauth2/authorize?code_challenge=_1eRtubYKCJiaQClZITt0_H7Wdt16fzSnW0Rf_aKag4&state=_default&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&code_challenge_method=S256&client_id=f8aa2b77-dafa-471d-af3b-5f5c6129eb82&scope=https%3A%2F%2Fauth.globus.org%2Fscopes%2Fga-test.tscwork.net%2Fssh&access_type=offline&response_type=code Please enter the code you get after login here: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 1 tscollins@tscollins-vm:~$ oauth-ssh -l tscollins ga-test.tscwork.net Last login: Thu Aug 29 20:38:52 2019 from 69.74.14.178 [tscollins@ga-test ~]$ id uid=1001(tscollins) gid=1002(tscollins) groups=1002(tscollins),4(adm),39(video),1000(google-sudoers) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [tscollins@ga-test ~]$

[JasonAlt]

Good. Please report back if you figure out what was wrong with the configuration on the ec2 instance.