After some benchmarking, it seems that setting the semaphore value to the same value as max_connections gives better performance and fewer timeout errors. If wanted, users can change the semaphore value via a keyword argument.
The plot below shows 5 different runs of Unparallel doing 2000 HTTP requests with varying max_connections (x-axis) and semaphore value which is defined by max_connections * semaphore_fact, i.e. for max_connections=300 and semaphore_fact=2 the semaphore value is 600. The y-axis shows the run time in seconds.
The data shows that if the semaphore value is greater than the maximum connections, the run time of finishing the 2000 requests increases drastically.
Additionally, some dependencies are upgraded due to CVEs. A policy config for safety is added to ignore a vulnerability in jinja2 that can't be fixed.
Changes
đ Fix timeouts for cases with lots of requests
âď¸ Fix typos and code fences in docstrings
đ Display latest alias in docs
âŹď¸ Upgrade anyio from 4.3.0 to 4.4.0 due to CVE
âŹď¸ Upgrade idna from 3.6 to 3.7 due to CVE
âŹď¸ Upgrade requests from 2.31.0 to 2.32.3 due to CVE
âŹď¸ Upgrade tqdm from 4.66.2 to 4.66.4 due to CVE
đ Add synchronization fix to Changelog
đď¸ Add safety policy file and ignore jinja2 vulnerability
Description
After some benchmarking, it seems that setting the semaphore value to the same value as max_connections gives better performance and fewer timeout errors. If wanted, users can change the semaphore value via a keyword argument.
The plot below shows 5 different runs of Unparallel doing 2000 HTTP requests with varying
max_connections
(x-axis) and semaphore value which is defined bymax_connections * semaphore_fact
, i.e. formax_connections=300
andsemaphore_fact=2
the semaphore value is600
. The y-axis shows the run time in seconds.The data shows that if the semaphore value is greater than the maximum connections, the run time of finishing the 2000 requests increases drastically.
Additionally, some dependencies are upgraded due to CVEs. A policy config for safety is added to ignore a vulnerability in jinja2 that can't be fixed.
Changes
Related Issue
Relates to #157
Type of Change
Checklist
CODE_OF_CONDUCT.md
document.CONTRIBUTING.md
guide.make codestyle
.