Failing github.ci run test

[ekes]

I'm not getting this locally, but it is occurring on all github run tests.

Run mkdir -p ./html/web/sites/simpletest && chmod 777 ./html/web/sites/simpletest
Running phpunit in 4 processes with /var/www/html/vendor/phpunit/phpunit/phpunit

Configuration read from /var/www/html/phpunit.xml.dist

.......................F.                                         25 / 25 (100%)

Time: 05:10.089, Memory: 10.00 MB

There was 1 failure:

1) Drupal\Tests\localgov_services_navigation\FunctionalJavascript\EntityReferenceServicesAutocompleteTest::testEntityReferenceAutocompleteWidget
Failed asserting that actual size 2 matches expected size 1.

/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:122
/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
/var/www/html/web/modules/contrib/localgov_services/modules/localgov_services_navigation/tests/src/FunctionalJavascript/EntityReferenceServicesAutocompleteTest.php:177
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

FAILURES!                               
Tests: 25, Assertions: 393, Failures: 1.
Error: Process completed with exit code 1.

[ekes]

The test is this https://github.com/localgovdrupal/localgov_services/blob/2.x/modules/localgov_services_navigation/tests/src/FunctionalJavascript/EntityReferenceServicesAutocompleteTest.php#L177 which would suggest that the browser still sees two results after 'Sub' has been written into the field. Something to do with the browser/webdriver would explain the difference?

[ekes]

I'm just going to log stuff I notice here. Running the tests locally (in the same containers as the github.ci) I get other fails:-

docker@767b77949263:/var/www/html$ vendor/phpunit/phpunit/phpunit web/modules/contrib/localgov_services --filter=EntityReferenceServicesAutocompleteTest                                                         
PHPUnit 9.5.28 by Sebastian Bergmann and contributors.

Testing /var/www/html/web/modules/contrib/localgov_services
FFE                                                                 3 / 3 (100%)

Time: 02:36.479, Memory: 12.00 MB

There was 1 error:

1) Drupal\Tests\localgov_services_navigation\FunctionalJavascript\EntityReferenceServicesAutocompleteTest::testEntityReferenceSublandingLabelWidget
Behat\Mink\Exception\ExpectationException: The field "edit-localgov-services-parent-0-target-id" value is "Page Sub 1 (2)", but "Landing Page 1 » Page Sub 1 (2)" expected.

/var/www/html/vendor/behat/mink/src/WebAssert.php:794
/var/www/html/vendor/behat/mink/src/WebAssert.php:701
/var/www/html/web/modules/contrib/localgov_services/modules/localgov_services_navigation/tests/src/FunctionalJavascript/EntityReferenceServicesAutocompleteTest.php:217
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

--

There were 2 failures:

1) Drupal\Tests\localgov_services_navigation\FunctionalJavascript\EntityReferenceServicesAutocompleteTest::testServicesEntityReferenceAutocompleteWidget
Failed asserting that actual size 0 matches expected size 1.

/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:122
/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
/var/www/html/web/modules/contrib/localgov_services/modules/localgov_services_navigation/tests/src/FunctionalJavascript/EntityReferenceServicesAutocompleteTest.php:97
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

2) Drupal\Tests\localgov_services_navigation\FunctionalJavascript\EntityReferenceServicesAutocompleteTest::testEntityReferenceAutocompleteWidget
Failed asserting that actual size 0 matches expected size 2.

/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:122
/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
/var/www/html/web/modules/contrib/localgov_services/modules/localgov_services_navigation/tests/src/FunctionalJavascript/EntityReferenceServicesAutocompleteTest.php:167
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

[ekes]

Run with screenshots in my local it's clear that it's not waiting at https://github.com/localgovdrupal/localgov_services/blob/2.x/modules/localgov_services_navigation/tests/src/FunctionalJavascript/EntityReferenceServicesAutocompleteTest.php#L173

[ekes]

With Drupal CI it seems it's possible to write the screenshots and then retrieve them from the test environment https://www.drupal.org/docs/automated-testing/phpunit-in-drupal/phpunit-javascript-test-writing-tutorial#s-screenshots. Do we know of being able to do something like this with github (before I research it)?

[finnlewis]

I don't know how to take screenshots in the Github CI, but I'd be suprised if it is not possible!

[ekes]

Argh!

In github and in lando

[ekes]

The only clue I can get from that is there is still a | after the Sub so maybe it's really just being so fast not even the blue spinner on the right has started?

Update yes it must be because it should have hit ' ' before that screenshot was made https://github.com/localgovdrupal/localgov_services/blob/85f11cb80e5e29ad9a019091718b3667c9c6bf56/modules/localgov_services_navigation/tests/src/FunctionalJavascript/EntityReferenceServicesAutocompleteTest.php#L172

[ekes]

Ah! It's because the autocomplete is already there! So waiting for it to appear doesn't wait for anything.