Open dualcode opened 3 months ago
gjb2048
commented
3 months ago But you don't write Behat tests for Adaptable because it's a theme. The overrides are there for when you run the core tests and set Adaptable as the theme, and they allow them to succeed due to the differences between it and Boost.
eric-s-richer
commented
3 months ago I've created a pull request https://github.com/gjbarnard/moodle-theme_adaptable/pull/31 to address this issue.
eric-s-richer
commented
3 months ago But you don't write Behat tests for Adaptable because it's a theme. The overrides are there for when you run the core tests and set Adaptable as the theme, and they allow them to succeed due to the differences between it and Boost.
You could absolutely write Behat tests to validate your theme's behaviour though. There's nothing preventing this from being done.
eric-s-richer
commented
3 months ago @gjb2048 as a follow up to your comment, consider the following in which I contrast a test run against the adaptable and default themes:
root@0991e34a996f:/var/www/html# php admin/tool/behat/cli/init.php --add-core-features-to-theme
You are already using the latest available Composer version 2.7.7 (stable channel). Installing dependencies from lock file (including require-dev) Verifying lock file contents can be installed on current platform. Nothing to install, update or remove Generating autoload files 53 packages you are using are looking for funding. Use thecomposer fundcommand to find out more! Behat test environment already installed Creating Behat configuration ... done in 0.38 seconds. Building theme CSS for adaptable [ltr] ... done in 1.03 seconds. Building theme CSS for boost [ltr] ... done in 0.91 seconds. Building theme CSS for classic [ltr] ... done in 0.93 seconds. Testing environment themes built Acceptance tests environment enabled on http://webserver, to run the tests use: vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --suite adaptable /var/www/html/login/tests/behat/navigation.feature Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070 Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64 Run optional tests: - Accessibility: Yes - SCSS deprecations: No Server OS "Linux", Browser: "firefox" Started at 09-08-2024, 07:17 FFFF \ --- Failed steps: \ 001 Scenario: Basic navigation without JavaScript # /var/www/html/login/tests/behat/navigation.feature:7 And I log in as "admin" # /var/www/html/login/tests/behat/navigation.feature:4 Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out() \
behat_theme_adaptable_behat_auth::i_log_out() behat_theme_adaptable_behat_auth::i_log_out()root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --suite default /var/www/html/login/tests/behat/navigation.feature Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070 Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64 Run optional tests: - Accessibility: Yes - SCSS deprecations: No Server OS "Linux", Browser: "firefox" Started at 09-08-2024, 07:18 .... \ 2 scenarios (2 passed) 4 steps (4 passed) 0m7.04s (52.40Mb)
In both instances, I am executing the login/tests/behat/navigation.feature Behat test whose contents are the exact same as the file in my pull request. Note that the tests fail against the adaptable suite; which is the theme.
gjb2048
commented
3 months ago Revisited, all very odd. Need to put on the backlog as not urgent.
eric-s-richer
commented
3 months ago @gjb2048 what's odd? If there's anything I can do to clarify or explain please let me know. I've literally written 100s of Behat tests for Moodle and various 3rd party plugins.
It would be really good to be able to write Behat tests which validate theme behaviour.
gjb2048
commented
3 months ago Odd in a way that's odd in relation to what has transpired. Will get back to this when can.
gjb2048
commented
3 months ago @eric-s-richer Note that your account for this issue has access to the development repository where you can see the latest code and will be a better place for the issue and any subsequent pull request(s).
dualcode
commented
3 months ago @gjb2048 I can now see your gjbarnard/moodle-theme_adaptable_dev repository but I am unable to fork it or submit pull requests. Am I not understanding the process here or am I maybe missing some permissions?
gjb2048
commented
3 months ago @gjb2048 I can now see your gjbarnard/moodle-theme_adaptable_dev repository but I am unable to fork it or submit pull requests. Am I not understanding the process here or am I maybe missing some permissions?
Try now please.
dualcode
commented
3 months ago It all seems to work now. As requested, I've opened a pull request.
gjb2048
commented
3 months ago Fix in dev, will close when release here.
Describe the bug If you were to write a Behat test for theme_adatpable, they'll fail due to the presense of contending step definitions in theme_adaptable.
To Reproduce Execute the feature file I'll provide in a branch and observe the failures. Note that all this test does it log in.
vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml /var/www/html/theme/adaptable/tests/behat/navigation.feature
Here is an example of such a failure:
`root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml /var/www/html/theme/adaptable/tests/behat/navigation.feature Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070 Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64 Run optional tests:
--- Failed steps:
001 Scenario: Basic navigation without JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:13 And I log in as "admin" # /var/www/html/theme/adaptable/tests/behat/navigation.feature:7 Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()
002 Scenario: Basic navigation without JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:13 Then I log out # /var/www/html/theme/adaptable/tests/behat/navigation.feature:11 Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()
003 Scenario: Basic navigation with JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:16 And I log in as "admin" # /var/www/html/theme/adaptable/tests/behat/navigation.feature:7 Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()
004 Scenario: Basic navigation with JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:16 Then I log out # /var/www/html/theme/adaptable/tests/behat/navigation.feature:11 Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()
2 scenarios (2 failed) 4 steps (4 failed) 0m7.32s (52.12Mb)`
Note that as you fix the root cause of one failure, you'll run into another. I dont' think it's neccessary to demonstrate every single failure I encountered, but let me know if you really need to see it.
Expected behavior A successful Behat run as follows: `root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml /var/www/html/theme/adaptable/tests/behat/navigation.feature Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070 Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64 Run optional tests:
2 scenarios (2 passed) 4 steps (4 passed) 0m8.84s (52.56Mb)
Versions (please complete the following information):
Additional context I'll link a branch with a fix for all of these in which the commit wil further explain the resolution.