amphp / byte-stream

A non-blocking stream abstraction for PHP based on Amp.
https://amphp.org/byte-stream
MIT License
363 stars 31 forks source link

Low bandwidth v2.0.0-beta.13, v2.0.0-beta.14 #103

Closed PNixx closed 1 year ago

PNixx commented 1 year ago

I use Ridge V2 for RabbitMQ. When I upgrade the package from version 12 to version v2.0.0-beta.14, message processing on the production server is greatly reduced (see point 1 in the figure). When I return version v2.0.0-beta.12 (see point 2 in the figure), the processing speed returns to the required value.

Снимок экрана 2022-12-28 в 16 12 56
Bilge commented 1 year ago

What about beta 13?

Bilge commented 1 year ago

If it is actually caused by beta 13, these are the changes. It seems that readable streams are now allowed to defer to other operations mid-stream. This shouldn't be an issue unless there are other things happening in your process besides reading and those things consume 100% of the CPU load or are synchronous.

PNixx commented 1 year ago

There are a lot of requests to PostgreSQL, HTTP requests, and PHPMailer calls via amphp/parallel are also used. The code does not use blocking threads. The thread limit in RabbitMQ is set to 20 per worker. CPU load is not more than 80%.

trowski commented 1 year ago

I think this may be due to optimization to avoid latency between a few reads in a row introduced in beta 13. Can you please try v2.x-dev (current 2.x branch) to see if https://github.com/amphp/byte-stream/commit/b5a09a3abd7a638ca4cadcf0876ce0ccd8d911d3 fixes the issue?

kelunik commented 1 year ago

@PNixx I can't reproduce any such behavior with the examples, so it'd be really helpful to hear your feedback for the commit mentioned above. Did you also update any other libraries?

PNixx commented 1 year ago

Checked on v2.0.0 - all works correctly

kelunik commented 1 year ago

@PNixx Thanks for confirming!

PNixx commented 1 year ago

After the update, I periodically catch errors in various microservices:

Uncaught Exception: Fiber stack protect failed: mprotect failed: Cannot allocate memory (12) in /home/shared/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php:479
Stack trace:
#0 /home/shared/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(479): Fiber->start()
#1 /home/shared/vendor/revolt/event-loop/src/EventLoop/Internal/AbstractDriver.php(533): Revolt\EventLoop\Internal\AbstractDriver->invokeCallbacks()
PNixx commented 1 year ago

I think the problems started after adding repeat reading https://github.com/amphp/byte-stream/compare/v2.0.0-beta.12...v2.0.0-beta.13

PNixx commented 1 year ago

@kelunik Very unstable code behavior. I had to downgrade the following packages: amphp/byte-stream: v2.0.0-beta.12 amphp/socket: v2.0.0-beta.7 amphp/dns: v2.0.0-beta.5 amphp/http-client: v5.0.0-beta.6 On the specified versions, the most stable and fastest processing.