Exception in third-party event subscriber: There is no active transaction

VincentLanglet commented 6 months ago

HI @dmaicher,

I'm recently getting the Exception in third-party event subscriber: There is no active transaction sometimes in my CI with this stracktrace:

#0 /var/www/wg-app/vendor/doctrine/dbal/src/Driver/PDO/Connection.php(137): Doctrine\DBAL\Driver\PDO\PDOException::new(Object(PDOException))
#1 /var/www/wg-app/vendor/dama/doctrine-test-bundle/src/DAMA/DoctrineTestBundle/Doctrine/DBAL/StaticDriver.php(69): Doctrine\DBAL\Driver\PDO\Connection->rollBack()
#2 /var/www/wg-app/vendor/dama/doctrine-test-bundle/src/DAMA/DoctrineTestBundle/PHPUnit/PHPUnitExtension.php(41): DAMA\DoctrineTestBundle\Doctrine\DBAL\StaticDriver::rollBack()
#3 /var/www/wg-app/vendor/dama/doctrine-test-bundle/src/DAMA/DoctrineTestBundle/PHPUnit/PHPUnitExtension.php(74): DAMA\DoctrineTestBundle\PHPUnit\PHPUnitExtension::rollBack()
#4 /var/www/wg-app/vendor/phpunit/phpunit/src/Event/Dispatcher/DirectDispatcher.php(102): PHPUnit\Event\Test\FinishedSubscriber@anonymous->notify(Object(PHPUnit\Event\Test\Finished))
#5 /var/www/wg-app/vendor/phpunit/phpunit/src/Event/Dispatcher/DeferringDispatcher.php([45](https://github.com/weglot/core/actions/runs/8751750775/job/24018022379?pr=5005#step:9:46)): PHPUnit\Event\DirectDispatcher->dispatch(Object(PHPUnit\Event\Test\Finished))
#6 /var/www/wg-app/vendor/phpunit/phpunit/src/Event/Emitter/DispatchingEmitter.php(998): PHPUnit\Event\DeferringDispatcher->dispatch(Object(PHPUnit\Event\Test\Finished))
#7 /var/www/wg-app/vendor/phpunit/phpunit/src/Framework/TestRunner.php(242): PHPUnit\Event\DispatchingEmitter->testFinished(Object(PHPUnit\Event\Code\TestMethod), 3)
#8 /var/www/wg-app/vendor/phpunit/phpunit/src/Framework/TestCase.php(488): PHPUnit\Framework\TestRunner->run(Object(App\Tests\Functional\Controller\Dashboard\SlugTranslationControllerTest))
#9 /var/www/wg-app/vendor/phpunit/phpunit/src/Framework/TestSuite.php(341): PHPUnit\Framework\TestCase->run()
#10 /var/www/wg-app/vendor/brianium/paratest/src/WrapperRunner/ApplicationForWrapperWorker.php(119): PHPUnit\Framework\TestSuite->run()
#11 /var/www/wg-app/vendor/brianium/paratest/bin/phpunit-wrapper.php(75): ParaTest\WrapperRunner\ApplicationForWrapperWorker->runTest('tests/Functiona...')
#12 /var/www/wg-app/vendor/brianium/paratest/bin/phpunit-wrapper.php(80): {closure}()

It occurs while running the tests, but

It doesn't occurs every time
I'm not sure it's related to a specific test.

I'm using Paratest https://github.com/paratestphp/paratest and I wonder if it would create some conflict.

I often solve the issue by re-running the ci. I wonder if try/catching the exception wouldn't be an "OK solution".

dmaicher commented 6 months ago

Does it happen without using paratest? I guess not?

How exactly does paratest run multiple tests in parallel? Is it happening in isolated processes? Or in the same php process?

VincentLanglet commented 6 months ago

Does it happen without using paratest? I guess not?

I'm sorry, kinda hard for me to answer this.

Our tests runs in ~10 minutes with paratest, without it would be around 30 minutes. And the bug occurs less than 1 times every 10 runs. So I would need to run the tests during hours to try to reproduce this without paratest, without proving anything :/

All I know is that

It occur with paratest
It seems recent, (I don't remember having this error one month ago - I got 2 weeks holidays) but it can also be related to the fact we have written more and more tests with the time.

How exactly does paratest run multiple tests in parallel? Is it happening in isolated processes? Or in the same php process?

According to https://github.com/paratestphp/paratest#initial-setup-for-all-tests "ParaTest runs multiple processes in parallel, each with their own instance of the PHP interpreter," does it answer your question ?

VincentLanglet commented 6 months ago

Not sure if it can helps but I also sometimes get the error

Doctrine\DBAL\Driver\PDO\Exception: SQLSTATE[42000]: Syntax error or access violation: 1305 SAVEPOINT DAMA_TEST does not exist

with the stack trace

/var/www/wg-app/vendor/doctrine/dbal/src/Driver/PDO/Connection.php:33
/var/www/wg-app/vendor/doctrine/dbal/src/Driver/Middleware/AbstractConnectionMiddleware.php:46
/var/www/wg-app/vendor/dama/doctrine-test-bundle/src/DAMA/DoctrineTestBundle/Doctrine/DBAL/StaticConnectionTrait.php:65
/var/www/wg-app/vendor/dama/doctrine-test-bundle/src/DAMA/DoctrineTestBundle/Doctrine/DBAL/StaticConnection.php:33
/var/www/wg-app/vendor/doctrine/dbal/src/Driver/Middleware/AbstractConnectionMiddleware.php:86
/var/www/wg-app/vendor/doctrine/dbal/src/Logging/Connection.php:80
/var/www/wg-app/vendor/doctrine/dbal/src/Driver/Middleware/AbstractConnectionMiddleware.php:86
/var/www/wg-app/vendor/symfony/doctrine-bridge/Middleware/Debug/DBAL3/Connection.php:127
/var/www/wg-app/vendor/doctrine/dbal/src/Connection.php:1521
/var/www/wg-app/vendor/doctrine/orm/src/UnitOfWork.php:485
/var/www/wg-app/vendor/doctrine/orm/src/EntityManager.php:403
/var/www/wg-app/src/Manager/AbstractManager.php:153

This seems definitely paratest-related because

The test failing is not always the same
I don't reproduce the error when I run paratest on a smaller folder of tests or when I run with phpunit

dmaicher commented 6 months ago

Are the different paratest processes using separate databases? Or a shared database?

VincentLanglet commented 6 months ago

Are the different paratest processes using separate databases? Or a shared database?

Shared database.

dmaicher commented 6 months ago

Really hard to tell :thinking:

Doctrine\DBAL\Driver\PDO\Exception: SQLSTATE[42000]: Syntax error or access violation: 1305 SAVEPOINT DAMA_TEST does not exist

Might be a sign that some query (doing DDL like alter table, truncate etc) implicitly committed a transaction which would mean that savepoint is gone.

VincentLanglet commented 6 months ago

Really hard to tell 🤔

Doctrine\DBAL\Driver\PDO\Exception: SQLSTATE[42000]: Syntax error or access violation: 1305 SAVEPOINT DAMA_TEST does not exist

Might be a sign that some query (doing DDL like alter table, truncate etc) implicitly committed a transaction which would mean that savepoint is gone.

The fact it doesn't occur every time (and on different tests) makes me think there is conflict between the parallel runner.

I tried to see if it was maybe because of the static properties... Then I just wonder it's because we can maybe have something like

Runner 1 start a savepoint
Runner 2 start a savepoint
Runner 1 end the savepoint clearing others too ?
Runner 2 doesnt have a savepoint anymore

And if this is the reason, I'm not sure I can do something about this concurrency, except having multiple databases...

dmaicher commented 6 months ago

I tried to see if it was maybe because of the static properties...

Hm yeah but I thought all runners have a separate php process? :thinking: then those static properties should be fine theoretically.

VincentLanglet commented 6 months ago

I tried to see if it was maybe because of the static properties...

Hm yeah but I thought all runners have a separate php process? 🤔 then those static properties should be fine theoretically.

Indeed, I just lost time of this idea ^^' So I think it should be the second option :/

VincentLanglet commented 6 months ago

I solve the issue by using multiple database.

dmaicher / doctrine-test-bundle

Exception in third-party event subscriber: There is no active transaction #290