pnpm / pacquet

experimental package manager for node.js
Apache License 2.0
762 stars 21 forks source link

feat(network): limit concurrent network requests #210

Closed KSXGitHub closed 9 months ago

KSXGitHub commented 9 months ago

Fix the "too many open files" error.

Closes https://github.com/pnpm/pacquet/pull/116

codecov[bot] commented 9 months ago

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (9a949c1) 87.02% compared to head (66ee2df) 87.04%. Report is 3 commits behind head on main.

Files Patch % Lines
crates/network/src/lib.rs 95.23% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #210 +/- ## ========================================== + Coverage 87.02% 87.04% +0.02% ========================================== Files 56 57 +1 Lines 2828 2848 +20 ========================================== + Hits 2461 2479 +18 - Misses 367 369 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 9 months ago

Micro-Benchmark Results

Linux

group                          main                                   pr
-----                          ----                                   --
tarball/download_dependency    1.01      6.7±0.21ms   647.1 KB/sec    1.00      6.7±0.21ms   651.5 KB/sec
github-actions[bot] commented 9 months ago

Integrated-Benchmark Report (Linux)

Scenario: Frozen Lockfile

Command Mean [ms] Min [ms] Max [ms] Relative
pacquet@HEAD 130.6 ± 5.9 119.6 139.2 1.00
pacquet@main 136.7 ± 10.9 123.7 162.4 1.05 ± 0.10
BENCHMARK_REPORT.json ```json { "results": [ { "command": "pacquet@HEAD", "mean": 0.13059700774666666, "stddev": 0.00585459595007108, "median": 0.12957050388, "user": 0.05845699333333334, "system": 0.2022172666666667, "min": 0.11955276388000001, "max": 0.13923595988, "times": [ 0.13419342588, 0.13785774088, 0.12945251088, 0.12941515588, 0.13923595988, 0.12854590288, 0.13115633288, 0.12646775388, 0.13298868988, 0.13832745488, 0.12715221688, 0.12059160388000001, 0.12957050388, 0.13444709988, 0.11955276388000001 ], "exit_codes": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ] }, { "command": "pacquet@main", "mean": 0.13670070330857143, "stddev": 0.010895159945852741, "median": 0.13691427088000002, "user": 0.058042945714285706, "system": 0.2071158, "min": 0.12373344788000003, "max": 0.16235738588, "times": [ 0.14308507388, 0.13171308388, 0.14737847088, 0.16235738588, 0.12478736688000001, 0.12782881288, 0.13901320088, 0.12390280088000001, 0.12373344788000003, 0.14531598588, 0.12971104388, 0.13544573388, 0.14115463088, 0.13838280788 ], "exit_codes": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ] } ] } ```
KSXGitHub commented 9 months ago

@zkochan I have tested this with a very big lockfile (which was generated from the alotta-files fixture) and see that it does emit any error while main still crashes with "too many open files". That is to say, this PR fixes the problem. Now, I want to ask if you want to add that big lockfile to the test?

zkochan commented 9 months ago

The big lockfile that I used for testing is created by this package.json: https://github.com/pnpm/pnpm.io/blob/main/benchmarks/fixtures/alotta-files/package.json