search

amphp / postgres

Async Postgres client for PHP based on Amp.
MIT License
97 stars 20 forks source link

[bug]: unexpected exception when pool limit is reached. #51

Closed azjezz closed 1 year ago

azjezz commented 2 years ago

given the following code:

<?php

use Amp\Postgres;
use Amp\Postgres\Link;
use Amp\Future;

require __DIR__ . '/../vendor/autoload.php';

$operate = static function (Link $pgsql, string $table, int $identifier): void {
    $pgsql->query('CREATE TABLE IF NOT EXISTS ' . $table . ' (id SERIAL PRIMARY KEY, content TEXT NOT NULL)');

    Future\all([
        Amp\async(static fn() => $pgsql->query('INSERT INTO ' . $table . ' ( content ) VALUES (\'Hello, You!\')')),
        Amp\async(static fn() => $pgsql->query('INSERT INTO ' . $table . ' ( content ) VALUES (\'Hello, City!\')')),
        Amp\async(static fn() => $pgsql->query('INSERT INTO ' . $table . ' ( content ) VALUES (\'Hello, Country!\')')),
        Amp\async(static fn() => $pgsql->query('INSERT INTO ' . $table . ' ( content ) VALUES (\'Hello, World!\')')),
        Amp\async(static fn() => $pgsql->query('INSERT INTO ' . $table . ' ( content ) VALUES (\'Hello, Galaxy!\')')),
        Amp\async(static fn() => $pgsql->query('INSERT INTO ' . $table . ' ( content ) VALUES (\'Hello, Universe!\')')),
    ]);

    $result = $pgsql->query('SELECT * FROM ' . $table . '; DROP TABLE IF EXISTS ' . $table);

    printf('[%*d] successfully operated on "%s" table, row count: %d%s', 3, $identifier, $table, $result->getRowCount(), "\n");
};

$config = Postgres\ConnectionConfig::fromString('host=127.0.0.1 port=32770 user=main password=main');
$link = Postgres\pool($config, maxConnections: 40);

$futures = [];
for ($i = 1; $i <= 40; $i++) {
    $table = uniqid('tmp_table_', false);
    $futures[] = Amp\async(fn() => $operate($link, $table, $i));
}

Future\all($futures);

I receive the following error:

> php example/database.php
[  1] successfully operated on "tmp_table_61f1e7eae9517" table, row count: 6
[  4] successfully operated on "tmp_table_61f1e7eae96a7" table, row count: 6
PHP Fatal error:  Uncaught Error: Operation is no longer pending in /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Internal/FutureState.php:88
Stack trace:
#0 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/DeferredFuture.php(29): Amp\Internal\FutureState->complete(Object(Amp\Postgres\PgSqlConnection))
#1 /home/azjezz/Projects/neutomic/neu/vendor/amphp/sql-common/src/ConnectionPool.php(311): Amp\DeferredFuture->complete(Object(Amp\Postgres\PgSqlConnection))
#2 /home/azjezz/Projects/neutomic/neu/vendor/amphp/sql-common/src/ConnectionPool.php(329): Amp\Sql\Common\ConnectionPool->push(Object(Amp\Postgres\PgSqlConnection))
#3 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(585): Amp\Sql\Common\ConnectionPool->Amp\Sql\Common\{closure}()
#4 [internal function]: Revolt\EventLoop\Internal\AbstractDriver::Revolt\EventLoop\Internal\{closure}()
#5 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(503): Fiber->resume(Array)
#6 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(414): Revolt\EventLoop\Internal\AbstractDriver->invokeMicrotasks()
#7 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Driver/StreamSelectDriver.php(285): Revolt\EventLoop\Internal\AbstractDriver->invokeCallback(Object(Revolt\EventLoop\Internal\StreamReadableCallback))
#8 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Driver/StreamSelectDriver.php(122): Revolt\EventLoop\Driver\StreamSelectDriver->selectStreams(Array, Array, 0.5556615100013)
#9 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(488): Revolt\EventLoop\Driver\StreamSelectDriver->dispatch(true)
#10 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(551): Revolt\EventLoop\Internal\AbstractDriver->tick()
#11 [internal function]: Revolt\EventLoop\Internal\AbstractDriver->Revolt\EventLoop\Internal\{closure}()
#12 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(100): Fiber->resume()
#13 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/DriverSuspension.php(80): Revolt\EventLoop\Internal\AbstractDriver->Revolt\EventLoop\Internal\{closure}()
#14 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Internal/FutureIterator.php(128): Revolt\EventLoop\Internal\DriverSuspension->suspend()
#15 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Future.php(56): Amp\Internal\FutureIterator->consume()
#16 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Future/functions.php(136): Amp\Future::iterate(Array, NULL)
#17 /home/azjezz/Projects/neutomic/neu/example/database.php(38): Amp\Future\all(Array)
#18 {main}
  thrown in /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Internal/FutureState.php on line 88

Fatal error: Uncaught Error: Operation is no longer pending in /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Internal/FutureState.php:88
Stack trace:
#0 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/DeferredFuture.php(29): Amp\Internal\FutureState->complete(Object(Amp\Postgres\PgSqlConnection))
#1 /home/azjezz/Projects/neutomic/neu/vendor/amphp/sql-common/src/ConnectionPool.php(311): Amp\DeferredFuture->complete(Object(Amp\Postgres\PgSqlConnection))
#2 /home/azjezz/Projects/neutomic/neu/vendor/amphp/sql-common/src/ConnectionPool.php(329): Amp\Sql\Common\ConnectionPool->push(Object(Amp\Postgres\PgSqlConnection))
#3 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(585): Amp\Sql\Common\ConnectionPool->Amp\Sql\Common\{closure}()
#4 [internal function]: Revolt\EventLoop\Internal\AbstractDriver::Revolt\EventLoop\Internal\{closure}()
#5 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(503): Fiber->resume(Array)
#6 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(414): Revolt\EventLoop\Internal\AbstractDriver->invokeMicrotasks()
#7 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Driver/StreamSelectDriver.php(285): Revolt\EventLoop\Internal\AbstractDriver->invokeCallback(Object(Revolt\EventLoop\Internal\StreamReadableCallback))
#8 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Driver/StreamSelectDriver.php(122): Revolt\EventLoop\Driver\StreamSelectDriver->selectStreams(Array, Array, 0.5556615100013)
#9 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(488): Revolt\EventLoop\Driver\StreamSelectDriver->dispatch(true)
#10 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(551): Revolt\EventLoop\Internal\AbstractDriver->tick()
#11 [internal function]: Revolt\EventLoop\Internal\AbstractDriver->Revolt\EventLoop\Internal\{closure}()
#12 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(100): Fiber->resume()
#13 /home/azjezz/Projects/neutomic/neu/vendor/revolt/event-loop/src/EventLoop/Internal/DriverSuspension.php(80): Revolt\EventLoop\Internal\AbstractDriver->Revolt\EventLoop\Internal\{closure}()
#14 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Internal/FutureIterator.php(128): Revolt\EventLoop\Internal\DriverSuspension->suspend()
#15 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Future.php(56): Amp\Internal\FutureIterator->consume()
#16 /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Future/functions.php(136): Amp\Future::iterate(Array, NULL)
#17 /home/azjezz/Projects/neutomic/neu/example/database.php(38): Amp\Future\all(Array)
#18 {main}
  thrown in /home/azjezz/Projects/neutomic/neu/vendor/amphp/amp/src/Internal/FutureState.php on line 88

Changing maxConnection to a higher number, or lower the number of operations ( e.g: $i <= 10 ), results in a successful run.

It appears that the lock mechanism used in the connection pool is incorrect, i haven't taken a deep look into it yet.

trowski commented 1 year ago

Fixed some time ago in https://github.com/amphp/sql-common/commit/08b00eb8e37383fba7b5ecbfd1adb1f8e3b795d2. Thanks as always for reporting!