Startup timeout on SAML login

[th122]

So I've been trying Chrome... The sandbox is using a password-based login (auth_type=Alma) and starts up successfully. The production environment is using SAML, and goes belly-up with a timeout:

Opening instance ...
Logging in as ...
Traceback (most recent call last):
  File "/usr/local/bin/slipsomat", line 11, in <module>
    load_entry_point('slipsomat==0.1.0', 'console_scripts', 'slipsomat')()
  File "build/bdist.macosx-10.11-x86_64/egg/slipsomat/slipsomat.py", line 829, in main
  File "build/bdist.macosx-10.11-x86_64/egg/slipsomat/slipsomat.py", line 168, in connect
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

In a fit, I've tried to "just double all those timeouts, whatever they may be good for", and I was able to complete the SAML login up to the functioning Alma interface, but it only took longer to die...

The SAML timeout used to be tight (using firefox while it was still working for me), and requied fast typing; but since the login has already happened when slipsomat dies, I don't think that's the issue here.

[danmichaelo]

Couldn't reproduce this using Chrome 58.0.3029.96, Chromedriver 2.29.461585 + Selenium 3.4.1.

Timeouts are generally symptoms of something else that went wrong.

[th122]

Chrome 58.0.3029.96 (64-bit), ChromeDriver 2.29.461585, Selenium 3.4.1. Switched to Python3 since you seem to be using that version and ended up with 3.6 (see #26). Still running into the timeout though. At least now the behavior during the SAML login is consistent across Firefox and Chrome. (It did work before the weekend...)

[danmichaelo]

Ok, so it fails to find an element with id "org" on the SAML login page:

https://github.com/scriptotek/alma-slipsomat/blob/master/slipsomat/slipsomat.py#L168

On my SAML login page this element is a dropdown menu:

I guess this could be implemented differently with different SAML providers. Perhaps there was some change to your SAML login page?

[th122]

I'm not aware of a change (but then it's not my domain to implement changes there), but slipsomat obviously is. An id-tag that seems to be present on both the Shibboleth login page as well as the unredirected Alma login page seems to be username or password, but that's an input-element. Looking at the source of our Shibboleth login page, there's not a single select-element in there, so it seems someone has been optimizing...

[th122]

I've been fumbling around a bit, and ended up with a working solution. I've targeted the login form as present on our current Shibboleth-page (sanitized to remove tooltips etc):

 <form action="/idp/profile/SAML2/Redirect/SSO;jsessionid=..." method="post">
     <div class="field">
         <label class="shibboleth" for="username">
                    HU-account                </label>
         <input class="login" size="15" name="j_username" id="username" tabindex="1" type="text" value="">
     </div>
     <div class="field">
         <label class="shibboleth" for="password">
                    HU-password                </label>
         <input class="login" size="15" name="j_password" id="password" tabindex="2" type="password" value="">
     </div>

     <!-- <input name="_eventId_proceed" tabindex="3" type="submit" value="Login"/> -->
     <button style = "margin-bottom: 1em; font-size: 140%;" type="submit" name="_eventId_proceed" value="Login" tabindex="3">Login</button>

</form>

And ended up with the following code-fragment:

        if auth_type == 'SAML':
            print('Logging in as {}@{}'.format(username, domain))

            element = wait.until(EC.visibility_of_element_located((By.ID, 'username')))
            element.send_keys(username)

            element = self.driver.find_element_by_id('password')
            element.send_keys(password)

            element = self.driver.find_element_by_name('_eventId_proceed')
            element.click()

[danmichaelo]

Yeah, but that would not work for me :)

Tried to push a commit now that could work for both. You can make domain empty in the config file and test if it works.

[th122]

I was afraid it wouldn't :) Unfortunately, the modified version with an empty domain in the config doesn't work for me either... it's still throwing that timeout exception.

[danmichaelo]

You did remove the domain from the config file, so it just says "Logging in as USER", not "Logging in as USER@DOMAIN"?

Which line is the timeout for now? If it's 165, then it doesn't find the "username" field, but that is strange considered the code you pasted above, so perhaps it's at some other line now?

[th122]

Clone of today's version - yes, it'sLogging in as username: (without domain), and neither username nor password get entered automagically. When I proceed by filling in the login form manually, it bails out, no matter if I'm fast and the Alma-page already got loaded, or it I'm taking my time to fill in username/password:

Traceback (most recent call last):
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/bin/slipsomat", line 11, in <module>
    sys.exit(main())
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/lib/python3.5/site-packages/slipsomat/slipsomat.py", line 815, in main
    browser.connect()
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/lib/python3.5/site-packages/slipsomat/slipsomat.py", line 175, in connect
    wait.until(EC.visibility_of_element_located((By.ID, 'ALMA_MENU_TOP_NAV_Search')))
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/lib/python3.5/site-packages/selenium-3.4.1-py3.5.egg/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 

[danmichaelo]

Hm, it's strange that it bails out on line 175 if the Alma-page already got loaded.. Anyways, could you try adding a few more debug statements like this:

        print('Waiting for username field')
        element = wait.until(EC.visibility_of_element_located((By.ID, 'username')))
        print('Sending username')
        element.send_keys(username)

        print('Waiting for password field')
        element = self.driver.find_element_by_id('password')
        print('Sending password')
        element.send_keys(password)

        print('Submitting')
        element.submit()

        print('Waiting for main alma page')
        try:
            # Look for some known element on the Alma main screen
            wait.until(EC.visibility_of_element_located((By.ID, 'ALMA_MENU_TOP_NAV_Search')))
        except NoSuchElementException:
            raise Exception('Failed to login to Alma')

If you don't help it by entering the credentials yourself, does it fail while waiting for the username field?

[th122]

fresh clone patched with the debug part you've sent above. First try: no manual interaction with browser window: Even though it claims to send username/password nothing gets entered in the browser window, and it is lolling a bit after Waiting for main alma page before the timeout kicks in. It may be that the page is just not loading fast enough when going through the manual input.

Opening instance ...
Logging in as <username>
Waiting for username field
Sending username
Waiting for password field
Sending password
Submitting
Waiting for main alma page
Traceback (most recent call last):
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/bin/slipsomat", line 11, in <module>
    sys.exit(main())
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/lib/python3.5/site-packages/slipsomat/slipsomat.py", line 821, in main
    browser.connect()
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/lib/python3.5/site-packages/slipsomat/slipsomat.py", line 181, in connect
    wait.until(EC.visibility_of_element_located((By.ID, 'ALMA_MENU_TOP_NAV_Search')))
  File "/Volumes/Users/theuser/.pyenv/versions/3.5.1/lib/python3.5/site-packages/selenium-3.4.1-py3.5.egg/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: 

... one more try: This time, I managed to get the newly creaeted browser window to the foreground fast enough to observe the entire buildup of the shibboleth login page: Username and password do get entered, but disappear immediately without leaving any visible trace on the page (a failed login would cause a message to be displayed within the page).

When exchanging the element.submit() for element.click() as in my example above, the login succeeds. The type of that (button) element is set to submit (quoted above) but that doesn't seem to be enough.

[danmichaelo]

Interesting. Does it work to replace

element.submit()

with this:

from selenium.webdriver.common.keys import Keys
element.send_keys(Keys.RETURN)

? Just trying to find something that is a little bit more general

[th122]

Brilliant idea, and it does send the login form off. Works perfectly for me!

[danmichaelo]

Great, added the fix in 729859f