Error on integration Test Magento 2.4.5

[francesco-di-marcantonio]

Hi this is the result when we run the integration test on a magento2.4.5

i cant find any reference for this error in magento.

Any suggestion?

Enabling caches:

In State.php line 213:

  [Magento\Framework\Exception\LocalizedException]
  Area code "frontend" does not exist

Exception trace:
  at /github/workspace/magento/vendor/magento/framework/App/State.php:213
 Magento\Framework\App\State->checkAreaCode() at /github/workspace/magento/vendor/magento/framework/App/State.php:133
 Magento\Framework\App\State->setAreaCode() at /github/workspace/magento/vendor/hyva-themes/magento2-theme-module/src/Model/HyvaModulesConfig.php:109
 Hyva\Theme\Model\HyvaModulesConfig->initFrontendArea() at /github/workspace/magento/vendor/hyva-themes/magento2-theme-module/src/Model/HyvaModulesConfig.php:102
 Hyva\Theme\Model\HyvaModulesConfig->createObjectManagerWithCurrentModulesList() at /github/workspace/magento/vendor/hyva-themes/magento2-theme-module/src/Model/HyvaModulesConfig.php:87
 Hyva\Theme\Model\HyvaModulesConfig->gatherConfigData() at /github/workspace/magento/vendor/hyva-themes/magento2-theme-module/src/Model/HyvaModulesConfig.php:51
 Hyva\Theme\Model\HyvaModulesConfig->getJson() at /github/workspace/magento/vendor/hyva-themes/magento2-theme-module/src/Model/HyvaModulesConfig.php:44
 Hyva\Theme\Model\HyvaModulesConfig->generateFile() at /github/workspace/magento/vendor/hyva-themes/magento2-theme-module/src/Plugin/HyvaModulesConfig/UpdateOnModuleStatusChange.php:40
 Hyva\Theme\Plugin\HyvaModulesConfig\UpdateOnModuleStatusChange->afterSaveConfig() at /github/workspace/magento/vendor/magento/framework/Interception/Interceptor.php:146
 Magento\Framework\App\DeploymentConfig\Writer\Interceptor->Magento\Framework\Interception\***closure***() at /github/workspace/magento/vendor/magento/framework/Interception/Interceptor.php:153
 Magento\Framework\App\DeploymentConfig\Writer\Interceptor->___callPlugins() at /github/workspace/magento/dev/tests/integration/tmp/sandbox-0-d773ce1a99a9bf5b8e2ac0f83a407f7b983d18ecd6acc5aeace3e6499b161009/generated/code/Magento/Framework/App/DeploymentConfig/Writer/Interceptor.php:32
 Magento\Framework\App\DeploymentConfig\Writer\Interceptor->saveConfig() at /github/workspace/magento/vendor/magento/framework/App/Cache/State.php:104
 Magento\Framework\App\Cache\State->persist() at /github/workspace/magento/vendor/magento/framework/App/Cache/Manager.php:77
 Magento\Framework\App\Cache\Manager->setEnabled() at /github/workspace/magento/setup/src/Magento/Setup/Model/Installer.php:1414
 Magento\Setup\Model\Installer->updateCaches() at n/a:n/a
 call_user_func_array() at /github/workspace/magento/setup/src/Magento/Setup/Model/Installer.php:386
 Magento\Setup\Model\Installer->install() at /github/workspace/magento/setup/src/Magento/Setup/Console/Command/InstallCommand.php:238
 Magento\Setup\Console\Command\InstallCommand->execute() at /github/workspace/magento/vendor/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /github/workspace/magento/vendor/symfony/console/Application.php:1009
 Symfony\Component\Console\Application->doRunCommand() at /github/workspace/magento/vendor/symfony/console/Application.php:273
 Symfony\Component\Console\Application->doRun() at /github/workspace/magento/vendor/magento/framework/Console/Cli.php:116
 Magento\Framework\Console\Cli->doRun() at /github/workspace/magento/vendor/symfony/console/Application.php:149
 Symfony\Component\Console\Application->run() at /github/workspace/magento/bin/magento:23

This is the file action:

name: m2-integration-test

on: [push]

jobs:
  magento2-integration-test:
    runs-on: ubuntu-latest
    container: ubuntu
    name: 'm2 integration test'
    services:
      mysql:
        image: docker://mysql:8
        env:
          MYSQL_ROOT_PASSWORD: magento
          MYSQL_DATABASE: magento
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=5 -e MYSQL_ROOT_PASSWORD=magento -e MYSQL_USER=magento -e MYSQL_PASSWORD=magento -e MYSQL_DATABASE=magento --entrypoint sh mysql:8 -c "exec docker-entrypoint.sh mysqld --default-authentication-plugin=mysql_native_password"
      elasticsearch:
        image: docker://elasticsearch:7.1.0
        ports:
          - 9200:9200
        options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
      rabbitmq:
        image: docker://rabbitmq:3.8-alpine
        env:
          RABBITMQ_DEFAULT_USER: "magento"
          RABBITMQ_DEFAULT_PASS: "magento"
          RABBITMQ_DEFAULT_VHOST: "/"
        ports:
          - 5672:5672

    steps:
      - uses: actions/checkout@v3
      - name: 'launch magento2 integration test'
        uses: MAD-I-T/magento-actions@v3.15
        env:
          COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
        with:
          php: '8.1'
          process: 'integration-test'
          elasticsearch: 1
          composer_version: 2

and this is the file mysql-config:

return [
    'db-host' => 'mysql',
    'db-user' => 'root',
    'db-password' => 'magento',
    'db-name' => 'magento',
    'db-prefix' => '',
    'backend-frontname' => 'backend',
    'admin-user' => \Magento\TestFramework\Bootstrap::ADMIN_NAME,
    'admin-password' => \Magento\TestFramework\Bootstrap::ADMIN_PASSWORD,
    'admin-email' => \Magento\TestFramework\Bootstrap::ADMIN_EMAIL,
    'admin-firstname' => \Magento\TestFramework\Bootstrap::ADMIN_FIRSTNAME,
    'admin-lastname' => \Magento\TestFramework\Bootstrap::ADMIN_LASTNAME,
    'amqp-host' => 'rabbitmq',
    'amqp-port' => '5672',
    'amqp-user' => 'magento',
    'amqp-password' => 'magento',
    'elasticsearch-host' => "elasticsearch",
    'elasticsearch-port' => "9200"
    ];

[seyuf]

Hi sir, I just tried this and seems to be working fine?:

name: m2-actions-test
on: [push]

jobs:
  magento2-integration-test:
    runs-on: ubuntu-latest
    container: ubuntu
    name: 'm2 integration test'
    services:
      mysql:
        image: docker://mysql:8
        env:
          MYSQL_ROOT_PASSWORD: magento
          MYSQL_DATABASE: magento
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=5 -e MYSQL_ROOT_PASSWORD=magento -e MYSQL_USER=magento -e MYSQL_PASSWORD=magento -e MYSQL_DATABASE=magento --entrypoint sh mysql:8 -c "exec docker-entrypoint.sh mysqld --default-authentication-plugin=mysql_native_password"
      elasticsearch:
        image: docker://elasticsearch:7.1.0
        ports:
          - 9200:9200
        options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
      rabbitmq:
        image: docker://rabbitmq:3.8-alpine
        env:
          RABBITMQ_DEFAULT_USER: "magento"
          RABBITMQ_DEFAULT_PASS: "magento"
          RABBITMQ_DEFAULT_VHOST: "/"
        ports:
          - 5672:5672

    steps:
      - name: 'install fresh magento repo'
        #if: always()
        if: ${{true}}
        uses: MAD-I-T/magento-actions@master
        env:
          COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
        with:
          process: 'create-project'
          magento_version: 2.4.5
          no_push: 1
      - name: 'launch magento2 integration test'
        if: ${{true}}
        uses: MAD-I-T/magento-actions@v3.15
        env:
          COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
        with:
          process: 'integration-test'
          elasticsearch: 1

[francesco-di-marcantonio]

tryed this file but it ends with an error. Any info?

[seyuf]

Yeah, magento integration tests, if you choose to do all of them, takes hours. So it is often killed by the actions after some time:
You should have some network warning/error like the following in the summary section of the workflow:

The hosted runner: Hosted Agent lost communication with the server. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.

[francesco-di-marcantonio]

How can i exclude some test? it looks like that it's not reading our phpunit.xml

[seyuf]

https://www.integer-net.com/integration-tests-with-magento-2/

Just try and modify dev/tests/integration/phpunit.xml.dist ? (not phpunit.xml)

[francesco-di-marcantonio]

ah okey, isn't there an option to change the phpunit file? Thanks for support!

[seyuf]

Humm you mean ? Something like dev/tests/integration/phpunit.xml ? instead? No but you're welcome to open a new issue for that feature. By the way, you can override the *integration-test-config*.php* files in the config dir through the override_settings flag

[seyuf]

Reopening this @bluerain-1 Update the doc for the posterity.

[seyuf]

Added support for auto selection of dev/tests/integration/phpunit.xml instead of dev/tests/integration/phpunit.xm.dist if exists. As well as testsuite, filter and test class path options, e.g below :

      - name: 'integration test by custom testsuite name'
        if: ${{true}}
        uses: MAD-I-T/magento-actions@master
        env:
          COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
        with:
          process: 'integration-test'
          elasticsearch: 1
          testsuite: "Memory Usage Tests"

      - name: 'integration test class'
        if: ${{true}}
        uses: MAD-I-T/magento-actions@master
        env:
          COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
        with:
          process: 'integration-test'
          elasticsearch: 1
          integration_class: "./testsuite/Magento/MemoryUsageTest.php"

      - name: 'integration test class filter'
        if: ${{true}}
        uses: MAD-I-T/magento-actions@master
        env:
          COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
        with:
          process: 'integration-test'
          elasticsearch: 1
          integration_class: "./testsuite/Magento/MemoryUsageTest.php"
          integration_filter: "testAppReinitializationNoMemoryLeak"

[francesco-di-marcantonio]

I will use this ticket because i dont know if this behaviour is related to the configuration or to the file. just for test we disabled every testuite commenting the phpunit.xml.dist

..II....PHP Warning:  Uncaught Error in exception handling during call to Error::__toString() in /github/workspace/magento/vendor/magento/framework/ObjectManager/Config/Config.php on line 173
PHP Stack trace:
PHP   1. ***main***() /github/workspace/magento/vendor/bin/phpunit:0
PHP   2. include() /github/workspace/magento/vendor/bin/phpunit:123
PHP   3. PHPUnit\TextUI\Command::main($exit = *uninitialized*) /github/workspace/magento/vendor/phpunit/phpunit/phpunit:98
PHP   4. PHPUnit\TextUI\Command->run($argv = [0 => '../../../vendor/bin/phpunit', 1 => '-c', 2 => 'phpunit.xml.dist'], $exit = TRUE) /github/workspace/magento/vendor/

or

PHP Warning:  Uncaught Error: Xdebug has detected a possible infinite loop, and aborted your script with a stack depth of '200' frames in /github/workspace/magento/vendor/magento/framework/ObjectManager/Config/Config.php:173
Stack trace:
#0 ***main***

  thrown in /github/workspace/magento/vendor/magento/framework/ObjectManager/Config/Config.php on line 173
PHP Stack trace:
PHP   1. ***main***() /github/workspace/magento/vendor/bin/phpunit:0
PHP   2. include() /github/workspace/magento/vendor/bin/phpunit:123
PHP   3. PHPUnit\TextUI\Command::main($exit = *uninitialized*) /github/workspace/magento/vendor/phpunit/phpunit/phpunit:98

PHP Fatal error:  __debuginfo() must return an array in /github/workspace/magento/vendor/magento/framework/ObjectManager/Config/Config.php on line 173
PHP Stack trace:
PHP   1. ***main***() /github/workspace/magento/vendor/bin/phpunit:0
PHP   2. include() /github/workspace/magento/vendor/bin/phpunit:123
PHP   3. PHPUnit\TextUI\Command::main($exit = *uninitialized*) /github/workspace/magento/vendor/phpunit/phpunit/phpunit:98
PHP   4. PHPUnit\TextUI\Command->run($argv = [0 => '../../../vendor/bin/phpunit', 1 => '-c', 2 => 'phpunit.xml.dist'], $exit = TRUE) /github/workspace/magento/vendor/phpunit/phpunit/src/TextUI/Command.php:97
PHP   5. PHPUnit\TextUI\TestRunner->run($suite = class PHPUnit\Framework\TestSuite *** protected $backupGlobals = NULL; protected $backupStaticAttributes = NULL; protected $runTestInSeparateProcess = FALSE; protected $name = ''; protected $groups = ['default' => [...], '__phpunit_covers_magento\authorization\model\rules::saverel' => [...], 

This is the file

<?xml version="1.0" encoding="UTF-8"?>
<!--
/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd"
         colors="true"
         columns="max"
         beStrictAboutTestsThatDoNotTestAnything="false"
         bootstrap="./framework/bootstrap.php"
         stderr="true"
         testSuiteLoaderClass="Magento\TestFramework\SuiteLoader"
         testSuiteLoaderFile="framework/Magento/TestFramework/SuiteLoader.php">
    <coverage>
        <include>

        </include>
        <exclude>
            <directory>../../../app/code/*/*/Test</directory>
            <directory>../../../lib/internal/*/*/Test</directory>
            <directory>../../../lib/internal/*/*/*/Test</directory>
            <directory>../../../setup/src/*/*/Test</directory>
            <!-- Exclude internal test magento2 -->
            <directory suffix=".php">../../../app/code/Magento</directory>
            <directory suffix=".php">../../../lib/internal/Magento</directory>
        </exclude>
    </coverage>
    <!-- Test suites definition -->
    <testsuites>
    <!--    <testsuite name="Magento Integration Tests">
            <file>testsuite/Magento/IntegrationTest.php</file>
        </testsuite>-->
        <!-- Memory tests run first to prevent influence of other tests on accuracy of memory measurements -->
     <!--   <testsuite name="Memory Usage Tests">
            <file>testsuite/Magento/MemoryUsageTest.php</file>
        </testsuite>-->
<!--        <testsuite name="Magento Integration Tests Real Suite">
            <directory>testsuite</directory>
            <directory>../../../app/code/*/*/Test/Integration</directory>
            <exclude>testsuite/Magento/MemoryUsageTest.php</exclude>
            <exclude>testsuite/Magento/IntegrationTest.php</exclude>
        </testsuite>-->
    </testsuites>
    <!-- Code coverage filters -->
    <!-- PHP INI settings and constants definition -->
    <php>
        <includePath>.</includePath>
        <includePath>testsuite</includePath>
        <ini name="date.timezone" value="America/Los_Angeles"/>
        <ini name="xdebug.max_nesting_level" value="200"/>
        <!-- Local XML configuration file ('.dist' extension will be added, if the specified file doesn't exist) -->
        <const name="TESTS_INSTALL_CONFIG_FILE" value="etc/install-config-mysql.php"/>
        <!-- Local XML post installation configuration file ('.dist' extension will be added, if the specified file doesn't exist) -->
        <const name="TESTS_POST_INSTALL_SETUP_COMMAND_CONFIG_FILE" value="etc/post-install-setup-command-config.php"/>
        <!-- Local XML configuration file ('.dist' extension will be added, if the specified file doesn't exist) -->
        <const name="TESTS_GLOBAL_CONFIG_FILE" value="etc/config-global.php"/>
        <!-- Semicolon-separated 'glob' patterns, that match global XML configuration files -->
        <const name="TESTS_GLOBAL_CONFIG_DIR" value="../../../app/etc"/>
        <!-- Whether to cleanup the application before running tests or not -->
        <const name="TESTS_CLEANUP" value="enabled"/>
        <!-- Memory usage and estimated leaks thresholds -->
        <!--<const name="TESTS_MEM_USAGE_LIMIT" value="1024M"/>-->
        <const name="TESTS_MEM_LEAK_LIMIT" value=""/>
        <!-- Path to Percona Toolkit bin directory -->
        <!--<const name="PERCONA_TOOLKIT_BIN_DIR" value=""/>-->
        <!-- CSV Profiler Output file -->
        <!--<const name="TESTS_PROFILER_FILE" value="profiler.csv"/>-->
        <!-- Bamboo compatible CSV Profiler Output file name -->
        <!--<const name="TESTS_BAMBOO_PROFILER_FILE" value="profiler.csv"/>-->
        <!-- Metrics for Bamboo Profiler Output in PHP file that returns array -->
        <!--<const name="TESTS_BAMBOO_PROFILER_METRICS_FILE" value="../../build/profiler_metrics.php"/>-->
        <!-- Whether to output all CLI commands executed by the bootstrap and tests -->
        <const name="TESTS_EXTRA_VERBOSE_LOG" value="1"/>
        <!-- Magento mode for tests execution. Possible values are "default", "developer" and "production". -->
        <const name="TESTS_MAGENTO_MODE" value="developer"/>
        <!-- Minimum error log level to listen for. Possible values: -1 ignore all errors, and level constants form http://tools.ietf.org/html/rfc5424 standard -->
        <const name="TESTS_ERROR_LOG_LISTENER_LEVEL" value="-1"/>
        <!-- Connection parameters for MongoDB library tests -->
        <!--<const name="MONGODB_CONNECTION_STRING" value="mongodb://localhost:27017"/>-->
        <!--<const name="MONGODB_DATABASE_NAME" value="magento_integration_tests"/>-->
        <!-- Connection parameters for RabbitMQ tests -->
        <!--<const name="RABBITMQ_MANAGEMENT_PROTOCOL" value="https"/>-->
        <!--<const name="RABBITMQ_MANAGEMENT_PORT" value="15672"/>-->
        <!--<const name="RABBITMQ_VIRTUALHOST" value="/"/>-->
        <!--<const name="TESTS_PARALLEL_RUN" value="1"/>-->
        <const name="USE_OVERRIDE_CONFIG" value="enabled"/>
    </php>
    <!-- Test listeners -->
    <listeners>
        <listener class="Yandex\Allure\PhpUnit\AllurePhpUnit">
            <arguments>
                <string>var/allure-results</string> <!-- XML files output directory -->
                <boolean>true</boolean> <!-- Whether to delete previous results on rerun -->
                <array> <!-- A list of custom annotations to ignore (optional) -->
                    <element key="codingStandardsIgnoreStart">
                        <string>codingStandardsIgnoreStart</string>
                    </element>
                    <element key="codingStandardsIgnoreEnd">
                        <string>codingStandardsIgnoreEnd</string>
                    </element>
                    <element key="expectedExceptionMessageRegExp">
                        <string>expectedExceptionMessageRegExp</string>
                    </element>
                    <element key="magentoAdminConfigFixture">
                        <string>magentoAdminConfigFixture</string>
                    </element>
                    <element key="magentoAppArea">
                        <string>magentoAppArea</string>
                    </element>
                    <element key="magentoAppIsolation">
                        <string>magentoAppIsolation</string>
                    </element>
                    <element key="magentoCache">
                        <string>magentoCache</string>
                    </element>
                    <element key="magentoComponentsDir">
                        <string>magentoComponentsDir</string>
                    </element>
                    <element key="magentoConfigFixture">
                        <string>magentoConfigFixture</string>
                    </element>
                    <element key="magentoDataFixture">
                        <string>magentoDataFixture</string>
                    </element>
                    <element key="magentoDataFixtureBeforeTransaction">
                        <string>magentoDataFixtureBeforeTransaction</string>
                    </element>
                    <element key="magentoDbIsolation">
                        <string>magentoDbIsolation</string>
                    </element>
                    <element key="magentoIndexerDimensionMode">
                        <string>magentoIndexerDimensionMode</string>
                    </element>
                </array>
            </arguments>
        </listener>
        <!-- Run after AllureAdapter to allow it to initialize properly -->
        <listener class="Magento\TestFramework\Event\PhpUnit"/>
        <listener class="Magento\TestFramework\ErrorLog\Listener"/>
    </listeners>
</phpunit>

[seyuf]

Hi @francesco-di-marcantonio see my preceding comment. You can now use dev/tests/integration/phpunit.xml (only tests in this file will be run if the file exists if not the .dist will run). Also no need to comment you can filter the integration run by the test suite name see here and the result in actions here. Let me know if that's good for you (if not i'll take a look at your above error probably a syntax issue? Also i don't get why you disabled all tests and then run a test?). BTW you have to use the @master version i'll publish a new version supporting these features ASAP.