search

pestphp / pest

Pest is an elegant PHP testing Framework with a focus on simplicity, meticulously designed to bring back the joy of testing in PHP.
https://pestphp.com
MIT License
9.42k stars 338 forks source link

Parallel testing does not stop on failure #381

Closed mertasan closed 3 years ago

mertasan commented 3 years ago 
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         ...
         stopOnFailure="true"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">

Next tests should not continue to run after the first failure. But it continues.

Is this a default behavior?

lukeraymonddowning commented 3 years ago

Question before I take this one; if you add another test error and run the suite, how many failures does it show, 2 or 1?

Reason I ask is that it might be stopping, but just emptying the processes it already has running.

mertasan commented 3 years ago

All tests are being processed.

Even if the first test fails, it continues until the last.

Result with failed tests:

   Tests: 5 failed, 267 passed
   Time: 25.39s

Result with passed tests: (without using parallel)

  Tests:  282 passed
  Time:   45.39s

@lukeraymonddowning Tests that fail when using parallel should actually have passed. This is another issue. (I made a detailed video about it. Please check your twitter DM.)

Some tests fail with parallel.

An example for failed tests:

Illuminate\View\ViewException 

File not found at path: T9L7m0ur9pnJuy0GCZyTuGn61SqiUx-metaaW1wb3J0YWJsZS5jc3Y=--size=430.csv (View: /Users/***/import-modal.blade.php) (View: /Users/***/import-modal.blade.php)
Illuminate\View\ViewException 

  include(/Users/***/vendor/orchestra/testbench-core/laravel/storage/framework/views/cd84f1c21074af6c34cce27e842eef993f72c65c.php): Failed to open stream: No such file or directory (View: /Users/***/tests/Data/resources/views/filter-results.blade.php)

  at vendor/livewire/livewire/src/ComponentConcerns/RendersLivewireComponents.php:71
mertasan commented 3 years ago

The errors that occurred were related to my package. For those who have the same problem, detailed explanation is here

Thanks for your help on this, @lukeraymonddowning .

This issue is only about stopOnFailure="true".