HumanSignal / Adala

Adala: Autonomous DAta (Labeling) Agent framework
https://humansignal.github.io/Adala/
Apache License 2.0
911 stars 72 forks source link

feat: PLT-396: Add gevent pooler #148

Open farioas opened 2 months ago

farioas commented 2 months ago

Let's say you need to execute thousands of HTTP GET requests to fetch data from external REST APIs. The time it takes to complete a single GET request depends almost entirely on the time it takes the server to handle that request. Most of the time, your tasks wait for the server to send the response, not using any CPU.

The bottleneck for this kind of task is not the CPU. The bottleneck is waiting for an Input/Output operation to finish. This is an Input/Output-bound task (I/O bound). The time the task takes to complete is determined by the time spent waiting for an input/output operation to finish.

If you run a single process execution pool, you can only handle one request at a time. It takes a long time to complete those thousands of GET requests. So you spawn more processes.

However, there is a tipping point where adding more processes to the execution pool hurts performance. The overhead of managing the process pool becomes more expensive than the marginal gain for an additional process.

In this scenario, spawning hundreds (or even thousands) of threads is a much more efficient way to increase capacity for I/O-bound tasks. Celery supports two thread-based execution pools: eventlet and gevent. Here, the execution pool runs in the same process as the Celery worker itself. To be precise, both eventlet and gevent use greenlets and not threads.

codecov-commenter commented 2 months ago

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 43.72%. Comparing base (91dc1d6) to head (e9d7644). Report is 3 commits behind head on master.

Files Patch % Lines
server/tasks/process_file.py 0.00% 4 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #148 +/- ## ========================================== - Coverage 43.79% 43.72% -0.08% ========================================== Files 40 40 Lines 1813 1816 +3 ========================================== Hits 794 794 - Misses 1019 1022 +3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/41eaca144f63dbd1b92256281b38d602c5ec641b is created.

robot-ci-heartex commented 2 months ago

Commit https://github.com/HumanSignal/infra/commit/6d192009d246e46b1d238a65715a8d1ef2cd0f50 is created.