huangdijia
commented
6 months ago Can you provide the method and steps to reproduce this problem?
Closed volodymyr-hordiienko closed 5 months ago
huangdijia
commented
6 months ago Can you provide the method and steps to reproduce this problem?
volodymyr-hordiienko
commented
6 months ago I need to reproduce issue on fresh project, need some time for it, version 3.16 works fine so issue i think in new HttpClient class and coroutine and channel inside. Currently i can only say I use hyperf/hyperf:8.2-alpine-v3.18-swoole-v5.1 docker image and following composer, deadlock happens each time i run command or other stuff and script try to be closed but coroutine inside HttpClient is sleeping:
{
"name": "hyperf/hyperf-skeleton",
"type": "project",
"keywords": [
"php",
"swoole",
"framework",
"hyperf",
"microservice",
"middleware"
],
"description": "A coroutine framework that focuses on hyperspeed and flexible, specifically use for build microservices and middlewares.",
"license": "Apache-2.0",
"require": {
"php": ">=8.2",
"ext-gd": "*",
"ext-openssl": "*",
"96qbhy/hyperf-auth": "^3.1",
"doctrine/lexer": "^2.0",
"fakerphp/faker": "^1.23",
"friendsofhyperf/sentry": "3.1.6",
"hyperf/amqp": "^3.1",
"hyperf/cache": "^3.1",
"hyperf/command": "^3.1",
"hyperf/config": "^3.1",
"hyperf/crontab": "^3.1",
"hyperf/database": "^3.1",
"hyperf/database-pgsql": "^3.1",
"hyperf/db-connection": "^3.1",
"hyperf/engine": "^2.10",
"hyperf/event": "^3.1",
"hyperf/framework": "^3.1",
"hyperf/guzzle": "^3.1",
"hyperf/http-server": "^3.1",
"hyperf/logger": "^3.1",
"hyperf/memory": "^3.1",
"hyperf/process": "^3.1",
"hyperf/redis": "^3.1",
"hyperf/resource": "^3.1",
"hyperf/translation": "^3.1",
"hyperf/validation": "^3.1",
"hyperf/view": "^3.01",
"hyperf/view-engine": "^3.1",
"kreait/firebase-php": "^7.9",
"yzen.dev/plain-to-class": "^3.0"
},
"require-dev": {
"hyperf/devtool": "^3.1",
"hyperf/testing": "^3.1",
"hyperf/watcher": "^3.1",
"mockery/mockery": "^1.6",
"slevomat/coding-standard": "^8.14",
"squizlabs/php_codesniffer": "^3.8",
"swoole/ide-helper": "^5.1",
"zircote/swagger-php": "^4.8"
},
"suggest": {
"ext-json": "Required to use JSON.",
"ext-pdo": "Required to use MySQL Client.",
"ext-pdo_mysql": "Required to use MySQL Client.",
"ext-redis": "Required to use Redis Client."
},
"autoload": {
"psr-4": {
"App\\": "app/"
},
"files": []
},
"autoload-dev": {
"psr-4": {
"HyperfTest\\": "./test/"
}
},
"minimum-stability": "dev",
"prefer-stable": true,
"config": {
"optimize-autoloader": true,
"sort-packages": true,
"allow-plugins": {
"php-http/discovery": true,
"dealerdirect/phpcodesniffer-composer-installer": true
}
},
"extra": [],
"scripts": {
"post-root-package-install": [
"@php -r \"file_exists('.env') || copy('.env.example', '.env');\""
],
"post-autoload-dump": [
"rm -rf runtime/container"
],
"cs-fix": "phpcbf --colors --standard=phpcs.xml ./app ./config",
"cs-check": "phpcs --colors --standard=phpcs.xml ./app ./config",
"test": ["php test/migrate.php && co-phpunit --prepend test/bootstrap.php -c phpunit.xml --colors=always"],
"swagger": "php ./bin/hyperf.php swagger:gen -o ./storage/public/documentation/",
"check-all": [
"@cs-fix",
"@cs-check",
"@test",
"@swagger"
],
"views-cache": "php ./bin/hyperf.php gen:view-engine-cache",
"migrate": "php ./bin/hyperf.php migrate",
"start": [
"Composer\\Config::disableProcessTimeout",
"php ./bin/hyperf.php start"
],
"watch": [
"Composer\\Config::disableProcessTimeout",
"php ./bin/hyperf.php server:watch"
]
}
}
Can you give a fake command code?
volodymyr-hordiienko
commented
6 months ago Just any code, even migrations, just tested simple command
php ./bin/hyperf.php test:command
<?php
declare(strict_types=1);
namespace App\Command;
use App\Service\Poll\TriggerPollsByDateService;
use App\Service\Poll\TriggerPollsByEventService;
use App\Service\PushNotification\SchedulePreRegistrationPushNotificationsService;
use App\Service\PushNotification\SendScheduledPushNotificationsService;
use Hyperf\Command\Annotation\Command;
use Hyperf\Command\Command as HyperfCommand;
use Psr\Container\ContainerInterface;
#[Command]
class TestCommand extends HyperfCommand
{
public function __construct()
{
parent::__construct('test:command');
}
public function configure(): void
{
parent::configure();
$this->setDescription('Test Command');
}
public function handle(): void
{
$this->info('Test Command');
}
}and got
grey-cardinal:/opt/www# php ./bin/hyperf.php test:command
[system_action][2024-01-11 22:38:36] app.DEBUG: [command] Commands registered by Hyperf\Command\Listener\RegisterCommandListener [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "Sentry\Integration\TransactionIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "Sentry\Integration\FrameContextifierIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "Sentry\Integration\EnvironmentIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "Sentry\Integration\ModulesIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "Sentry\Integration\RequestIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "FriendsOfHyperf\Sentry\Integration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "FriendsOfHyperf\Sentry\Integration\ExceptionContextIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "FriendsOfHyperf\Sentry\Integration\RequestIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: The "FriendsOfHyperf\Sentry\Integration\TraceIntegration" integration has been installed. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: Crontab Sync Wp Task have been registered. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: Crontab Trigger Polls By Date Task have been registered. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: Crontab Send Scheduled Push Notifications Task have been registered. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: Crontab Schedule No SMS Push Notifications Task have been registered. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: Crontab Truncate Old Database Data Task have been registered. [] []
[system_action][2024-01-11 22:38:36] app.DEBUG: Crontab Parse Active Personal Offers Task have been registered. [] []
Test Command
===================================================================
[FATAL ERROR]: all coroutines (count: 2) are asleep - deadlock!
===================================================================
[Coroutine-3]
--------------------------------------------------------------------
#0 /opt/www/vendor/hyperf/engine/src/Channel.php(41): Swoole\Coroutine\Channel->pop()
#1 /opt/www/vendor/hyperf/coordinator/src/Coordinator.php(34): Hyperf\Engine\Channel->pop()
#2 /opt/www/vendor/friendsofhyperf/sentry/src/HttpClient/HttpClient.php(96): Hyperf\Coordinator\Coordinator->yield()
#3 [internal function]: FriendsOfHyperf\Sentry\HttpClient\HttpClient->FriendsOfHyperf\Sentry\HttpClient\{closure}()
[Coroutine-2]
--------------------------------------------------------------------
#0 /opt/www/vendor/hyperf/engine/src/Channel.php(41): Swoole\Coroutine\Channel->pop()
#1 /opt/www/vendor/friendsofhyperf/sentry/src/HttpClient/HttpClient.php(74): Hyperf\Engine\Channel->pop()
#2 [internal function]: FriendsOfHyperf\Sentry\HttpClient\HttpClient->FriendsOfHyperf\Sentry\HttpClient\{closure}()
[2024-01-11 22:38:37 @572.0] WARNING Channel::~Channel() (ERRNO 10003): channel is destroyed, 1 consumers will be discarded
[2024-01-11 22:38:37 @572.0] WARNING Channel::~Channel() (ERRNO 10003): channel is destroyed, 1 consumers will be discardedits important that sentry dsn config was in place, if its empty its not reproduce for me
volodymyr-hordiienko
commented
6 months ago Just tried to debug HttpClient
Coroutine::create(function () {
if (CoordinatorManager::until(Constants::WORKER_EXIT)->yield()) {
var_dump('HERE!');
$this->close();
}
});and nothing output, seems Constants::WORKER_EXIT not firing for me...
volodymyr-hordiienko
commented
6 months ago Here my sentry config if it can help, i'll continue to debug
<?php
declare(strict_types=1);
/**
* This file is part of friendsofhyperf/components.
*
* @link https://github.com/friendsofhyperf/components
* @document https://github.com/friendsofhyperf/components/blob/main/README.md
* @contact huangdijia@gmail.com
*/
use FriendsOfHyperf\Sentry\Integration;
use function Hyperf\Support\env;
return [
'dsn' => env('SENTRY_DSN', ''),
// The release version of your application
// Example with dynamic git hash: trim(exec('git --git-dir ' . base_path('.git') . ' log --pretty="%h" -n1 HEAD'))
'release' => env('SENTRY_RELEASE'),
// When left empty or `null` the Laravel environment will be used (usually discovered from `APP_ENV` in your `.env`)
'environment' => env('APP_ENV', 'production'),
// @see: https://docs.sentry.io/platforms/php/guides/laravel/configuration/options/#sample-rate
'sample_rate' => env('SENTRY_SAMPLE_RATE') === null
? 1.0
: (float)env('SENTRY_SAMPLE_RATE'),
// Switch tracing on/off
'enable_tracing' => env('SENTRY_ENABLE_TRACING', true),
// @see: https://docs.sentry.io/platforms/php/guides/laravel/configuration/options/#traces-sample-rate
'traces_sample_rate' => env('SENTRY_TRACES_SAMPLE_RATE') === null
? null
: (float)env('SENTRY_TRACES_SAMPLE_RATE'),
// @see: https://docs.sentry.io/platforms/php/guides/laravel/configuration/options/#profiles-sample-rate
'profiles_sample_rate' => env('SENTRY_PROFILES_SAMPLE_RATE') === null
? null
: (float)env('SENTRY_PROFILES_SAMPLE_RATE'),
// @see: https://docs.sentry.io/platforms/php/guides/laravel/configuration/options/#send-default-pii
'send_default_pii' => env('SENTRY_SEND_DEFAULT_PII', false),
'logger' => Hyperf\Contract\StdoutLoggerInterface::class,
'enable' => [
'amqp' => env('SENTRY_ENABLE_AMQP', true),
'async_queue' => env('SENTRY_ENABLE_ASYNC_QUEUE', true),
'command' => env('SENTRY_ENABLE_COMMAND', true),
'kafka' => env('SENTRY_ENABLE_KAFKA', true),
'request' => env('SENTRY_ENABLE_REQUEST', true),
],
'breadcrumbs' => [
'sql_queries' => env('SENTRY_BREADCRUMBS_SQL_QUERIES', true),
'sql_bindings' => env('SENTRY_BREADCRUMBS_SQL_BINDINGS', true),
'sql_transaction' => env('SENTRY_BREADCRUMBS_SQL_TRANSACTION', true),
'redis' => env('SENTRY_BREADCRUMBS_REDIS', true),
'guzzle' => env('SENTRY_BREADCRUMBS_GUZZLE', true),
'logs' => env('SENTRY_BREADCRUMBS_LOGS', true),
],
'integrations' => [
Integration\TraceIntegration::class,
],
'ignore_exceptions' => [
App\Exception\BusinessException::class,
Hyperf\Database\Model\ModelNotFoundException::class,
Hyperf\Database\Model\RelationNotFoundException::class,
Hyperf\HttpMessage\Exception\NotFoundHttpException::class,
Hyperf\HttpMessage\Exception\BadRequestHttpException::class,
Hyperf\HttpMessage\Exception\ForbiddenHttpException::class,
Hyperf\HttpMessage\Exception\HttpException::class,
Hyperf\HttpMessage\Exception\MethodNotAllowedHttpException::class,
Hyperf\HttpMessage\Exception\NotAcceptableHttpException::class,
Hyperf\HttpMessage\Exception\NotFoundHttpException::class,
Hyperf\HttpMessage\Exception\RangeNotSatisfiableHttpException::class,
Hyperf\HttpMessage\Exception\ServerErrorHttpException::class,
Hyperf\HttpMessage\Exception\UnauthorizedHttpException::class,
Hyperf\HttpMessage\Exception\UnprocessableEntityHttpException::class,
Hyperf\HttpMessage\Exception\UnsupportedMediaTypeHttpException::class,
Hyperf\Validation\ValidationException::class,
Qbhy\SimpleJwt\Exceptions\JWTException::class,
Qbhy\HyperfAuth\Exception\UnauthorizedException::class,
],
'ignore_transactions' => [
'GET /health',
],
// Performance monitoring specific configuration
'tracing' => [
'enable' => [
'coroutine' => env('SENTRY_TRACING_ENABLE_COROUTINE', true),
'db' => env('SENTRY_TRACING_ENABLE_DB', true),
'elasticsearch' => env('SENTRY_TRACING_ENABLE_ELASTICSEARCH', true),
'guzzle' => env('SENTRY_TRACING_ENABLE_GUZZLE', true),
'rpc' => env('SENTRY_TRACING_ENABLE_RPC', true),
'redis' => env('SENTRY_TRACING_ENABLE_REDIS', true),
'sql_queries' => env('SENTRY_TRACING_ENABLE_SQL_QUERIES', true),
],
'tags' => [
'annotation' => [
'coroutine.id' => 'coroutine.id',
'arguments' => 'arguments',
// 'result' => 'result',
],
'coroutine' => [
'id' => 'coroutine.id',
],
'db' => [
'coroutine.id' => 'coroutine.id',
'query' => 'db.query',
// 'result' => 'db.result',
],
'elasticsearch' => [
'coroutine.id' => 'coroutine.id',
'arguments' => 'arguments',
// 'result' => 'result',
],
'guzzle' => [
'coroutine.id' => 'coroutine.id',
'http.method' => 'http.method',
'http.uri' => 'http.uri',
'guzzle.config' => 'guzzle.config',
'request.options' => 'request.options',
'response.status' => 'response.status',
'response.reason' => 'response.reason',
'response.headers' => 'response.headers',
],
'redis' => [
'coroutine.id' => 'coroutine.id',
'pool' => 'pool',
'arguments' => 'arguments',
// 'result' => 'result',
],
'rpc' => [
'coroutine.id' => 'coroutine.id',
'arguments' => 'arguments',
// 'result' => 'result',
],
'sql_queries' => [
'coroutine.id' => 'coroutine.id',
'db.connection_name' => 'db.connection_name',
'db.bindings' => 'db.bindings',
],
],
],
];I tried to create a new project and could not reproduce your problem.
huangdijia
commented
6 months ago <?php
declare(strict_types=1);
/**
* This file is part of friendsofhyperf/components.
*
* @link https://github.com/friendsofhyperf/components
* @document https://github.com/friendsofhyperf/components/blob/main/README.md
* @contact huangdijia@gmail.com
*/
namespace FriendsOfHyperf\Sentry\HttpClient;
use Closure;
use Hyperf\Coordinator\Constants;
use Hyperf\Coordinator\CoordinatorManager;
use Hyperf\Coroutine\Concurrent;
use Hyperf\Engine\Channel;
use Hyperf\Engine\Coroutine;
use Sentry\HttpClient\Request;
use Sentry\HttpClient\Response;
use Sentry\Options;
use Throwable;
class HttpClient extends \Sentry\HttpClient\HttpClient
{
protected ?Channel $chan = null;
protected ?Concurrent $concurrent = null;
public function __construct(
string $sdkIdentifier,
string $sdkVersion,
protected int $channelSize = 65535,
int $concurrentLimit = 100,
) {
parent::__construct($sdkIdentifier, $sdkVersion);
if ($concurrentLimit > 0) {
$this->concurrent = new Concurrent($concurrentLimit);
}
}
public function sendRequest(Request $request, Options $options): Response
{
$this->loop();
$chan = $this->chan;
$chan->push(fn () => parent::sendRequest($request, $options));
return new Response(202, [], 'Waiting for sendRequest');
}
public function close(): void
{
$chan = $this->chan;
$this->chan = null;
$chan?->close();
}
protected function loop(): void
{
if ($this->chan != null) {
return;
}
$this->chan = new Channel($this->channelSize);
Coroutine::create(function () {
while (true) {
while (true) {
+ if (! $this->chan?->isAvailable()) {
+ break 2;
+ }
/** @var Closure|null $closure */
$closure = $this->chan?->pop();
if (! $closure || ! $closure instanceof Closure) {
break 2;
}
try {
if ($this->concurrent) {
$this->concurrent->create($closure);
} else {
Coroutine::create($closure);
}
} catch (Throwable) {
break;
} finally {
$closure = null;
}
}
}
$this->close();
});
Coroutine::create(function () {
if (CoordinatorManager::until(Constants::WORKER_EXIT)->yield()) {
$this->close();
}
});
}
}
Try adding these lines of code to see if it solves your problem.
or add the Hyperf\Coordinator\Listener\ResumeExitCoordinatorListener::class to config/autoload/listeners.php.
volodymyr-hordiienko
commented
5 months ago Yeah! Add the Hyperf\Coordinator\Listener\ResumeExitCoordinatorListener::class to config/autoload/listeners.php fix the issue, thank you a lot for your support with this! Think it would be helpful to add this config to documentation
fayno-market-grey-cardinal.app | =================================================================== fayno-market-grey-cardinal.app | [FATAL ERROR]: all coroutines (count: 2) are asleep - deadlock! fayno-market-grey-cardinal.app | =================================================================== fayno-market-grey-cardinal.app | fayno-market-grey-cardinal.app | [Coroutine-3] fayno-market-grey-cardinal.app | -------------------------------------------------------------------- fayno-market-grey-cardinal.app | #0 /opt/www/vendor/hyperf/engine/src/Channel.php(41): Swoole\Coroutine\Channel->pop() fayno-market-grey-cardinal.app | #1 /opt/www/vendor/hyperf/coordinator/src/Coordinator.php(34): Hyperf\Engine\Channel->pop() fayno-market-grey-cardinal.app | #2 /opt/www/vendor/friendsofhyperf/sentry/src/HttpClient/HttpClient.php(96): Hyperf\Coordinator\Coordinator->yield() fayno-market-grey-cardinal.app | #3 [internal function]: FriendsOfHyperf\Sentry\HttpClient\HttpClient->FriendsOfHyperf\Sentry\HttpClient{closure}() fayno-market-grey-cardinal.app | fayno-market-grey-cardinal.app | fayno-market-grey-cardinal.app | [Coroutine-2] fayno-market-grey-cardinal.app | -------------------------------------------------------------------- fayno-market-grey-cardinal.app | #0 /opt/www/vendor/hyperf/engine/src/Channel.php(41): Swoole\Coroutine\Channel->pop() fayno-market-grey-cardinal.app | #1 /opt/www/vendor/friendsofhyperf/sentry/src/HttpClient/HttpClient.php(74): Hyperf\Engine\Channel->pop() fayno-market-grey-cardinal.app | #2 [internal function]: FriendsOfHyperf\Sentry\HttpClient\HttpClient->FriendsOfHyperf\Sentry\HttpClient{closure}()