magento / magento2-functional-testing-framework

Magento2 Functional Testing Framework
Other
155 stars 132 forks source link

v4.3.4 - Empty exception messages thrown after test failure #910

Open snez opened 1 year ago

snez commented 1 year ago

Preconditions

  1. PHP 8.2
  2. Magento 2.4.6
  3. MFTF 4.3.4
  4. Selenium 3.141.59

Steps to reproduce

  1. Have a test fail

Expected result

  1. A screenshot should be successfully attached to the reports

Actual result

  1. Execution fails with an empty exception message, and all subsequent tests do not run

Example:

[clickPlaceOrder] ClickPlaceOrderActionGroup
  [waitForPlaceOrderButton] wait for element ".payment-method._active button.action.primary.checkout",30
  [saveScreenshot] save screenshot
  [] [START AFTER HOOK]
[] [logoutStorefront] StorefrontCustomerLogoutActionGroup
  [storefrontSignOut] am on page "customer/account/logout/"
  [waitForSignOut] wait for page load 60
[deleteProduct] delete entity "createSimpleProduct","hook"
[deleteCategory] delete entity "createCategory","hook"
[deleteCustomer] delete entity "createCustomer","hook"
[END AFTER HOOK]
 ERROR

/******** Beginning execution of AllTestsSuite suite after block ********/
LAST TEST IN SUITE FAILED, TEST AFTER MAY NOT BE SUCCESSFUL

/******** Execution of AllTestsSuite suite after block complete ********/
------------------------------------------------------------

In AllureHelper.php line 29:

  [Exception]

Exception trace:
  at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Allure/AllureHelper.php:29
 Magento\FunctionalTestingFramework\Allure\AllureHelper::addAttachmentToCurrentStep() at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php:333
 Magento\FunctionalTestingFramework\Extension\TestContextExtension->attachExceptionToAllure() at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php:142
 Magento\FunctionalTestingFramework\Extension\TestContextExtension->testEnd() at /var/www/html/vendor/symfony/event-dispatcher/EventDispatcher.php:220
 Symfony\Component\EventDispatcher\EventDispatcher->callListeners() at /var/www/html/vendor/symfony/event-dispatcher/EventDispatcher.php:56
 Symfony\Component\EventDispatcher\EventDispatcher->dispatch() at /var/www/html/vendor/codeception/codeception/src/Codeception/Test/Test.php:238
 Codeception\Test\Test->realRun() at /var/www/html/vendor/codeception/codeception/src/Codeception/Suite.php:130
 Codeception\Suite->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/SuiteManager.php:148
 Codeception\SuiteManager->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Codecept.php:260
 Codeception\Codecept->runSuite() at /var/www/html/vendor/codeception/codeception/src/Codeception/Codecept.php:216
 Codeception\Codecept->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Command/Run.php:646
 Codeception\Command\Run->runSuites() at /var/www/html/vendor/codeception/codeception/src/Codeception/Command/Run.php:467
 Codeception\Command\Run->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/html/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Application.php:112
 Codeception\Application->run() at /var/www/html/vendor/codeception/codeception/app.php:45
 {closure}() at /var/www/html/vendor/codeception/codeception/app.php:46
 require() at /var/www/html/vendor/codeception/codeception/codecept:7
 include() at /var/www/html/vendor/bin/codecept:119

The buggy code is at:

     public static function addAttachmentToCurrentStep($data, $caption): void
    {
        if (!is_string($data)) {
            try {
                $data = serialize($data);
            } catch (\Exception $exception) {
                throw  new \Exception($data->getMessage());
            }
        }
        if (@file_exists($data) && is_file($data)) {
            Allure::attachmentFile($caption, $data);
        } else {
            Allure::attachment($caption, $data);
        }
    }
m2-assistant[bot] commented 1 year ago

Hi @snez. Thank you for your report. To speed up processing of this issue, make sure that you provided sufficient information. Add a comment to assign the issue: @magento I am working on this


Join Magento Community Engineering Slack and ask your questions in #github channel.

snez commented 1 year ago

I tweaked the buggy method to reveal the exception message, and it was:

Serialization of 'Closure' is not allowed