liuggio / fastest

Simple parallel testing execution... with some goodies for functional tests.
MIT License
475 stars 65 forks source link

Failure on several issues #173

Closed mnsami closed 3 years ago

mnsami commented 3 years ago

I'm running a laravel 5.3 and php7.2 project, and I'm trying to use fastest to boil down a 30mins test run.

My tests are divided between:

  1. System
  2. Integration
  3. Unit

each are defined inside a group in phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
        bootstrap="bootstrap/autoload.php"
        stopOnFailure="true"
        colors="true"
>
    <testsuites>
        <testsuite name="Unit">
            <directory>./tests/Unit</directory>
        </testsuite>
        <testsuite name="System">
            <directory>./tests/System</directory>
        </testsuite>
        <testsuite name="API">
            <directory>./tests/Unit</directory>
            <directory>./tests/System</directory>
        </testsuite>
        <testsuite name="Integration">
            <directory>./tests/Integration</directory>
        </testsuite>
    </testsuites>
    <php>
        <env name="APP_ENV" value="testing"/>
        <env name="BASE_URL" value="https://application.localhost"/>
        <env name="DB_CONNECTION" value="db_testing"/>
    </php>
</phpunit>

But there is something I can't wrap my head around, the env variables.

When I run find tests/ -name "*Test.php" | vendor/bin/fastest -o "vendor/bin/phpunit -c phpunit.xml {};" -vv I get errors like:

any help or pointers about what am I doing wrong ?

In our base SystemTest we are setting the db by running migrations and seeding.

Any help would be appreciated.

DonCallisto commented 3 years ago

It sounds like is a concurrency issue, not fastest one. Maybe your tests are not independant so, when parallelized, the issue arise.

mnsami commented 3 years ago

@DonCallisto you mean each test function within a class ?

I have to deep investigate that, it is a company project I would like to optimize.

But can't the fastest env variables fix that in a way ?

DonCallisto commented 3 years ago

As stated before, I don't think it's a matter of fastest. Maybe your app is trying to aquire a lock on something. When you execute tests with phpunit (sequentially), you won't notice this, but when more processes are running at the same time, you'll end up in concurrent acquisition attempt. Maybe it's a spy for possible issue inside the app itself. Don't know why you're referring to env variables here: have they something to do with locks or serialization or whatever?

mnsami commented 3 years ago

thanks @DonCallisto .. I have solved some of the issues by debugging.