search

pestphp / pest

Pest is an elegant PHP testing Framework with a focus on simplicity, meticulously designed to bring back the joy of testing in PHP.
https://pestphp.com
MIT License
9.63k stars 357 forks source link

RefreshDatabase unstable in mixed PHPUnit / Pest setup #593

Closed zenlex closed 2 years ago

zenlex commented 2 years ago

On two separate projects I've run into this issue recently. The Laravel projects in question have mixed test suites with some PHPUnit and some PEST tests, all being run together via artisan test. The PHPUnit tests contain the use RefreshDatabase trait. The Pest tests all have uses(RefreshDatabase::class) bound globally in the Pest.php file.

When running the full feature test suite, sometimes tests that were passing now fail and vice versa without functional code changes. Re-running the tests multiple times will cause the failure to move or disappear altogether. It appears the database refresh is not stable/reliable in this setup. Refactoring the PHPUnit to Pest or vice versa appears to eliminate the issue, as does preventing either the Pest or the PHPUnit tests from refreshing the DB. Running the failing tests in isolation with --filter also makes them pass when this is occurring.

Is there some variance in how the DB transactions/locks/migrations are handled with the PHPUnit tests vs Pest that could be getting out of sync here?

DB is Postgres. Laravel 9, Pest 1.21, pest-plugin-laravel 1.1

nunomaduro commented 2 years ago

Can you create a project on GitHub where we can reproduce the issue, and re-open an issue here?

zenlex commented 1 year ago

I'll try when I get a minute to see if I can repro on a project I can share.

masiorama commented 1 year ago

Having the same issue.

titantwentyone commented 1 year ago

@zenlex did you create an example repo? I am experiencing the same issue with the exact same set up (mixture of PHPUnit and Pest) and wondered if a solution/explanation had been found.

zenlex commented 1 year ago

Haven't had time to set up a complex enough example to repro the problem yet and on the project where we had the issue we just refactored all the php unit tests to Pest and the problem went away. Have experienced it on more than one project though. No problem mixing php unit and pest as long as they don't both use the refreshDatabase trait.

agustinprod commented 1 year ago

Having the same issue but no mixed tests 🤔