Open Andarist opened 2 days ago
I do like running tests
@typescript-bot test it
Hey @jakebailey, the results of running the DT tests are ready.
Everything looks the same!
@jakebailey Here are the results of running the user tests with tsc comparing main
and refs/pull/60557/merge
:
Something interesting changed - please have a look.
@jakebailey Here are some more interesting changes from running the user tests suite
@jakebailey The results of the perf run you requested are in!
Metric | baseline | pr | Delta | Best | Worst | p-value |
---|---|---|---|---|---|---|
Compiler-Unions - node (v18.15.0, x64) | ||||||
Errors | 34 | 34 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 62,363 | 62,363 | ~ | ~ | ~ | p=1.000 n=6 |
Types | 50,395 | 50,395 | ~ | ~ | ~ | p=1.000 n=6 |
Memory used | 193,640k (± 0.76%) | 193,096k (± 0.08%) | ~ | 193,005k | 193,410k | p=0.810 n=6 |
Parse Time | 1.31s (± 0.89%) | 1.31s (± 0.96%) | ~ | 1.29s | 1.32s | p=0.801 n=6 |
Bind Time | 0.72s | 0.72s (± 0.57%) | ~ | 0.71s | 0.72s | p=0.405 n=6 |
Check Time | 9.84s (± 0.35%) | 9.79s (± 0.39%) | -0.05s (- 0.49%) | 9.74s | 9.84s | p=0.036 n=6 |
Emit Time | 2.72s (± 0.55%) | 2.73s (± 0.28%) | ~ | 2.72s | 2.74s | p=0.318 n=6 |
Total Time | 14.58s (± 0.30%) | 14.54s (± 0.31%) | ~ | 14.48s | 14.59s | p=0.089 n=6 |
angular-1 - node (v18.15.0, x64) | ||||||
Errors | 37 | 37 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 947,936 | 947,937 | +1 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
Types | 410,955 | 410,959 | +4 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
Memory used | 1,226,009k (± 0.00%) | 1,225,996k (± 0.00%) | ~ | 1,225,949k | 1,226,066k | p=0.689 n=6 |
Parse Time | 6.65s (± 0.64%) | 6.64s (± 0.78%) | ~ | 6.59s | 6.70s | p=0.809 n=6 |
Bind Time | 1.89s (± 0.22%) | 1.88s (± 0.56%) | ~ | 1.87s | 1.90s | p=0.528 n=6 |
Check Time | 31.93s (± 0.44%) | 31.92s (± 0.57%) | ~ | 31.65s | 32.09s | p=0.936 n=6 |
Emit Time | 15.22s (± 0.34%) | 15.21s (± 0.26%) | ~ | 15.15s | 15.27s | p=0.935 n=6 |
Total Time | 55.68s (± 0.23%) | 55.66s (± 0.41%) | ~ | 55.35s | 55.84s | p=0.809 n=6 |
mui-docs - node (v18.15.0, x64) | ||||||
Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 2,499,513 | 2,499,513 | ~ | ~ | ~ | p=1.000 n=6 |
Types | 909,291 | 909,291 | ~ | ~ | ~ | p=1.000 n=6 |
Memory used | 2,317,335k (± 0.00%) | 2,317,326k (± 0.00%) | ~ | 2,317,273k | 2,317,383k | p=0.689 n=6 |
Parse Time | 9.25s (± 0.19%) | 9.25s (± 0.20%) | ~ | 9.22s | 9.27s | p=0.685 n=6 |
Bind Time | 2.16s (± 0.74%) | 2.16s (± 0.68%) | ~ | 2.14s | 2.18s | p=1.000 n=6 |
Check Time | 75.07s (± 0.57%) | 74.79s (± 0.36%) | -0.28s (- 0.37%) | 74.54s | 75.31s | p=0.045 n=6 |
Emit Time | 0.29s (± 1.92%) | 0.28s (± 3.64%) | ~ | 0.27s | 0.30s | p=0.663 n=6 |
Total Time | 86.77s (± 0.49%) | 86.49s (± 0.30%) | -0.28s (- 0.33%) | 86.25s | 87.00s | p=0.045 n=6 |
self-build-src - node (v18.15.0, x64) | ||||||
Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 1,225,251 | 1,225,276 | +25 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
Types | 266,545 | 266,550 | +5 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
Memory used | 2,354,196k (± 0.01%) | 2,414,811k (± 6.13%) | ~ | 2,353,747k | 2,717,488k | p=0.471 n=6 |
Parse Time | 5.21s (± 0.61%) | 5.23s (± 1.25%) | ~ | 5.17s | 5.36s | p=1.000 n=6 |
Bind Time | 1.78s (± 1.05%) | 1.77s (± 1.27%) | ~ | 1.74s | 1.79s | p=0.566 n=6 |
Check Time | 35.27s (± 0.32%) | 35.11s (± 0.43%) | ~ | 34.83s | 35.23s | p=0.066 n=6 |
Emit Time | 2.95s (± 1.49%) | 2.99s (± 1.05%) | ~ | 2.94s | 3.03s | p=0.077 n=6 |
Total Time | 45.23s (± 0.29%) | 45.12s (± 0.18%) | ~ | 44.98s | 45.21s | p=0.148 n=6 |
self-build-src-public-api - node (v18.15.0, x64) | ||||||
Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 1,225,251 | 1,225,276 | +25 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
Types | 266,545 | 266,550 | +5 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
Memory used | 2,908,165k (±12.85%) | 3,149,364k (± 0.01%) | ~ | 3,148,606k | 3,149,703k | p=0.575 n=6 |
Parse Time | 6.98s (± 1.89%) | 7.00s (± 0.85%) | ~ | 6.94s | 7.09s | p=1.000 n=6 |
Bind Time | 2.16s (± 1.61%) | 2.12s (± 1.35%) | ~ | 2.08s | 2.16s | p=0.092 n=6 |
Check Time | 42.66s (± 0.60%) | 42.73s (± 0.31%) | ~ | 42.62s | 42.97s | p=0.471 n=6 |
Emit Time | 3.47s (± 2.09%) | 3.52s (± 1.31%) | ~ | 3.46s | 3.58s | p=0.199 n=6 |
Total Time | 55.26s (± 0.68%) | 55.38s (± 0.30%) | ~ | 55.16s | 55.68s | p=0.520 n=6 |
self-compiler - node (v18.15.0, x64) | ||||||
Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 262,223 | 262,223 | ~ | ~ | ~ | p=1.000 n=6 |
Types | 106,606 | 106,607 | +1 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
Memory used | 439,769k (± 0.01%) | 439,795k (± 0.01%) | ~ | 439,737k | 439,833k | p=0.471 n=6 |
Parse Time | 3.55s (± 1.81%) | 3.55s (± 0.68%) | ~ | 3.52s | 3.59s | p=0.628 n=6 |
Bind Time | 1.32s (± 1.70%) | 1.32s (± 0.31%) | ~ | 1.31s | 1.32s | p=0.548 n=6 |
Check Time | 18.99s (± 0.39%) | 18.98s (± 0.40%) | ~ | 18.88s | 19.09s | p=1.000 n=6 |
Emit Time | 1.52s (± 1.02%) | 1.51s (± 0.69%) | ~ | 1.50s | 1.53s | p=0.621 n=6 |
Total Time | 25.39s (± 0.51%) | 25.37s (± 0.33%) | ~ | 25.24s | 25.50s | p=0.936 n=6 |
ts-pre-modules - node (v18.15.0, x64) | ||||||
Errors | 70 | 70 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 226,062 | 226,062 | ~ | ~ | ~ | p=1.000 n=6 |
Types | 94,488 | 94,488 | ~ | ~ | ~ | p=1.000 n=6 |
Memory used | 371,669k (± 0.03%) | 371,694k (± 0.04%) | ~ | 371,553k | 371,967k | p=1.000 n=6 |
Parse Time | 2.91s (± 0.86%) | 2.89s (± 1.16%) | ~ | 2.84s | 2.94s | p=0.294 n=6 |
Bind Time | 1.59s (± 1.03%) | 1.60s (± 1.50%) | ~ | 1.57s | 1.63s | p=0.411 n=6 |
Check Time | 16.48s (± 0.32%) | 16.50s (± 0.23%) | ~ | 16.46s | 16.55s | p=0.746 n=6 |
Emit Time | 0.00s (±244.70%) | 0.00s (±244.70%) | ~ | 0.00s | 0.01s | p=1.000 n=6 |
Total Time | 20.97s (± 0.28%) | 20.98s (± 0.19%) | ~ | 20.94s | 21.04s | p=0.687 n=6 |
vscode - node (v18.15.0, x64) | ||||||
Errors | 4 | 4 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 3,163,934 | 3,163,934 | ~ | ~ | ~ | p=1.000 n=6 |
Types | 1,090,167 | 1,090,167 | ~ | ~ | ~ | p=1.000 n=6 |
Memory used | 3,244,420k (± 0.02%) | 3,244,419k (± 0.02%) | ~ | 3,243,742k | 3,244,838k | p=0.810 n=6 |
Parse Time | 14.08s (± 0.72%) | 14.07s (± 0.52%) | ~ | 13.96s | 14.17s | p=1.000 n=6 |
Bind Time | 4.58s (± 2.51%) | 4.50s (± 0.44%) | ~ | 4.47s | 4.52s | p=0.252 n=6 |
Check Time | 88.31s (± 1.86%) | 87.92s (± 3.86%) | ~ | 85.33s | 92.36s | p=0.471 n=6 |
Emit Time | 26.04s (± 9.46%) | 26.94s (± 7.93%) | ~ | 22.87s | 28.91s | p=0.630 n=6 |
Total Time | 133.02s (± 1.21%) | 133.44s (± 1.86%) | ~ | 131.14s | 137.97s | p=1.000 n=6 |
webpack - node (v18.15.0, x64) | ||||||
Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 288,617 | 288,617 | ~ | ~ | ~ | p=1.000 n=6 |
Types | 117,107 | 117,107 | ~ | ~ | ~ | p=1.000 n=6 |
Memory used | 440,829k (± 0.03%) | 440,861k (± 0.01%) | ~ | 440,804k | 440,952k | p=0.471 n=6 |
Parse Time | 4.06s (± 0.55%) | 4.07s (± 1.21%) | ~ | 4.01s | 4.13s | p=0.627 n=6 |
Bind Time | 1.75s (± 1.02%) | 1.74s (± 0.47%) | ~ | 1.73s | 1.75s | p=0.324 n=6 |
Check Time | 18.88s (± 0.42%) | 18.87s (± 0.30%) | ~ | 18.79s | 18.96s | p=0.748 n=6 |
Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
Total Time | 24.69s (± 0.42%) | 24.69s (± 0.37%) | ~ | 24.55s | 24.81s | p=0.747 n=6 |
xstate-main - node (v18.15.0, x64) | ||||||
Errors | 8 | 8 | ~ | ~ | ~ | p=1.000 n=6 |
Symbols | 551,583 | 551,583 | ~ | ~ | ~ | p=1.000 n=6 |
Types | 184,808 | 184,808 | ~ | ~ | ~ | p=1.000 n=6 |
Memory used | 491,882k (± 0.00%) | 491,855k (± 0.03%) | ~ | 491,512k | 491,962k | p=0.173 n=6 |
Parse Time | 3.41s (± 1.18%) | 3.40s (± 0.49%) | ~ | 3.38s | 3.42s | p=0.871 n=6 |
Bind Time | 1.17s (± 0.90%) | 1.17s (± 0.70%) | ~ | 1.16s | 1.18s | p=0.865 n=6 |
Check Time | 19.54s (± 0.38%) | 19.69s (± 1.50%) | ~ | 19.46s | 20.28s | p=0.261 n=6 |
Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
Total Time | 24.12s (± 0.25%) | 24.26s (± 1.17%) | ~ | 24.03s | 24.82s | p=0.298 n=6 |
Benchmark | Name | Iterations |
---|---|---|
Current | pr | 6 |
Baseline | baseline | 6 |
@jakebailey Here are the results of running the top 400 repos with tsc comparing main
and refs/pull/60557/merge
:
Something interesting changed - please have a look.
Redux-based repro from the reports above: TS playground. Interestingly, it actually fails to be inferred with current version and it does infer with this change... but then it fails to typecheck. The current version works because it tries out the last compose
's overload which is very permissive.
fp-ts
' change: TS playground. It's an error position change.
fixes https://github.com/microsoft/TypeScript/issues/60552
I'm not exactly sure if this is right but the presented issue shows how the current logic can lead to surprising results. The existing test cases are fairly OK with this change, it would be great to run the extended test suite to learn what kind of unwanted effects this change might have (cc @jakebailey )