search

catalyst / moodle-auth_saml2

SAML done 100% in Moodle, fast, simple, secure
https://moodle.org/plugins/auth_saml2
72 stars 134 forks source link

Behat tests are broken after MDL-72173 #607

Closed dmitriim closed 3 years ago

dmitriim commented 3 years ago

As part of https://tracker.moodle.org/browse/MDL-72173 log out step was changed in https://github.com/moodle/moodle/blob/master/auth/tests/behat/behat_auth.php#L74

This causes following tests to fail:

001 Scenario: SAML2 logout                                                                    # /home/runner/work/moodle-auth_saml2/moodle-auth_saml2/moodle/auth/saml2/tests/behat/login.feature:34
      And the mock SAML IdP confirms logout                                      # auth_saml2 # /home/runner/work/moodle-auth_saml2/moodle-auth_saml2/moodle/auth/saml2/tests/behat/login.feature:47
        Not on the IdP logout page.

        +--[ http://localhost:8000/ | WebDriver ]
        |
        |  <html dir="ltr" lang="en" xml:lang="en" class="yui3-js-enabled"><head>
        |      <title>Acceptance test site</title>
        |      <link rel="shortcut icon" href="http://localhost:8000/theme/image.php/boost/theme/1632293742/favicon">
        |      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        |  <meta name="keywords" content="moodle, Acceptance test site">
        |  <link rel="stylesheet" type="text/css" href="http://localhost:8000/theme/yui_combo.php?rollup/3.17.2/yui-moodlesimple.css"><script charset="utf-8" id="yui_3_17_2_1_1632294247791_8" src="http://localhost:8000/theme/yui_combo.php?m/1632293742/core/event/event-debug.js&amp;m/1632293742/filter_mathjaxloader/loader/loader-debug.js" async=""></script><script charset="utf-8" id="yui_3_17_2_1_1632294247791_20" src="http://localhost:8000/theme/yui_combo.php?3.17.2/event-mousewheel/event-mousewheel.js&amp;3.17.2/event-resize/event-resize.js&amp;3.17.2/event-hover/event-hover.js&amp;3.17.2/event-touch/event-touch.js&amp;3.17.2/event-move/event-move...
        |

1 scenario (1 failed)
11 steps (9 passed, 1 failed, 1 skipped)
0m14.04s (54.19Mb)
To re-run failed processes, you can use following command:
php admin/tool/behat/cli/run.php -vvv --profile="chrome" --tags="@auth_saml2" --suite="default" --rerun
andrewnicols commented 3 years ago

Since this Scenario is testing the log out behaviour you should avoid using a step like “I log out” anyway. Where you are testing a specific behaviour you should avoid using custom steps to test that behaviour too much because they obscure failures and are prone to being rewritten as things which do not use the UI.

I’d also advise against writing a custom wrapper around the click handler - you should just be able to use:

When I click on “Log out” “link” in the “#page-footer” “css_element”

I need to double-check the exact usage (and possibly add an xpath selector for it) but this should really be in the “footer” “region” too.

Also a word of warning: The step you have written is very generic in @Given (should be a @When technically) - steps are shared by all uses in Moodle. If someone else writes a step like this (or you do in another plugin) then both will match.

There’s also no need for the pending_js check - that’s built in to all steps in the Moodle behat extension.

dmitriim commented 3 years ago

thanks @andrewnicols Good point. I'll update my patch.

danmarsden commented 3 years ago

thanks @andrewnicols !