pnpm / pacquet

experimental package manager for node.js
Apache License 2.0
762 stars 21 forks source link

test: filter out index files temporarily #197

Closed KSXGitHub closed 10 months ago

KSXGitHub commented 10 months ago

Changes in the npm registry cause unnecessary snapshot failures in our tests. The ideal solution is to implement a fake registry that we can control, but until then, filtering them out is good enough.

github-actions[bot] commented 10 months ago

Micro-Benchmark Results

Linux

group                          main                                   pr
-----                          ----                                   --
tarball/download_dependency    1.00      6.7±0.20ms   644.2 KB/sec    1.03      7.0±1.73ms   622.9 KB/sec
codecov[bot] commented 10 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6b00165) 87.11% compared to head (d031d26) 87.04%. Report is 1 commits behind head on main.

:exclamation: Current head d031d26 differs from pull request most recent head 0e8bfe6. Consider uploading reports for the commit 0e8bfe6 to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #197 +/- ## ========================================== - Coverage 87.11% 87.04% -0.07% ========================================== Files 56 56 Lines 2824 2825 +1 ========================================== - Hits 2460 2459 -1 - Misses 364 366 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 10 months ago

Integrated-Benchmark Report (Linux)

Scenario: Frozen Lockfile

Command Mean [ms] Min [ms] Max [ms] Relative
pacquet@HEAD 138.4 ± 8.4 123.3 147.8 1.05 ± 0.09
pacquet@main 132.3 ± 7.2 123.4 145.5 1.00
BENCHMARK_REPORT.json ```json { "results": [ { "command": "pacquet@HEAD", "mean": 0.1384241537542857, "stddev": 0.008414157122340154, "median": 0.13771208354, "user": 0.05704961714285715, "system": 0.20423031999999997, "min": 0.12327296254000002, "max": 0.14780967154, "times": [ 0.14737982154, 0.14552210354, 0.13390590354, 0.14518258154, 0.13540762154, 0.13246162254, 0.14723337954, 0.13342041354, 0.14634528454, 0.13896461154, 0.14780967154, 0.12327296254000002, 0.13645955554, 0.12457261954 ], "exit_codes": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ] }, { "command": "pacquet@main", "mean": 0.13227419422750003, "stddev": 0.0071780807654990625, "median": 0.13035875654, "user": 0.06165876, "system": 0.20150538250000002, "min": 0.12335731054000001, "max": 0.14548627454, "times": [ 0.12544221254, 0.13664241354, 0.12335731054000001, 0.14035412653999998, 0.12445625754000002, 0.12903680554, 0.12357289854000002, 0.14548627454, 0.12660023154, 0.13906317354, 0.13065172953999998, 0.13006578354, 0.14343481353999998, 0.13496635854, 0.12831084154, 0.13494587654 ], "exit_codes": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ] } ] } ```
zkochan commented 10 months ago

How does this mitigate the issue? The snapshot will be invalid if any of the files from the package will change. Or a package will get a new file.

KSXGitHub commented 10 months ago

How does this mitigate the issue?

Create an actual mock registry that we have full control over. Or maybe just a verdaccio proxy is sufficient?

The snapshot will be invalid if any of the files from the package will change.

Right now, the package versions are pinned. This does mean that we cannot test something involving version ranges (such as pacquet add foo@latest), so eventually, we would have to properly solve the problem.

zkochan commented 10 months ago

For pnpm we use a verdaccio proxy with prepublished packages for testing: https://github.com/pnpm/registry-mock

I think we should use the same in this repo.