search

kkebo / zyphy

Zyphy is (or will be) a fast web browser engine written in Swift.
MIT License
4 stars 1 forks source link

ci: enable parallel test #128

Closed kkebo closed 5 days ago

kkebo commented 5 days ago

I added --parallel to swift test. This improves performance of test execution. I would like to enable it if it doesn't affect code coverage.

github-actions[bot] commented 5 days ago

Coverage Summary

Sat Nov 9 14:18:14 UTC 2024

Filename                           Functions  Missed Functions  Executed       Lines      Missed Lines     Cover
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
HTMLEntities/namedChars.swift              1                 0   100.00%          12                 0   100.00%
Str/Str.swift                             24                 7    70.83%          64                19    70.31%
Tokenizer/BufferQueue.swift               11                 1    90.91%          33                 1    96.97%
Tokenizer/CharRefTokenizer.swift          17                 0   100.00%         227                 4    98.24%
Tokenizer/DOCTYPE.swift                    2                 0   100.00%          14                 0   100.00%
Tokenizer/Optional+take.swift              1                 0   100.00%           5                 0   100.00%
Tokenizer/SmallCharSet.swift               4                 0   100.00%           8                 0   100.00%
Tokenizer/Tag.swift                        1                 0   100.00%           6                 0   100.00%
Tokenizer/Tokenizer.swift                121                14    88.43%        1297               218    83.19%
Tokenizer/Utils.swift                      2                 0   100.00%          13                 0   100.00%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                    184                22    88.04%        1679               242    85.59%
kkebo commented 5 days ago

According to https://github.com/swiftlang/swift-package-manager/issues/7069, it's ok to use --parallel with --enable-code-coverage.

github-actions[bot] commented 5 days ago

Benchmark Summary

Sat Nov 9 14:26:41 UTC 2024

==============================================
Threshold deviations for MyBenchmark:lipsum-zh
==============================================
Time (wall clock) (μs, %) main pull_request Difference % Threshold %
p50 1488 1567 5 5
p75 1524 1620 6 5 
===================================================
Threshold deviations for MyBenchmark:small-fragment
===================================================
Time (wall clock) (μs, %) main pull_request Difference % Threshold %
p50 4514 4743 5 5
p75 4530 4792 5 5 
==================================================
Threshold deviations for MyBenchmark:tiny-fragment
==================================================
Time (wall clock) (μs, %) main pull_request Difference % Threshold %
p25 445 471 5 5
p50 448 479 7 5
p75 450 482 7 5

New baseline 'pull_request' is WORSE than the 'main' baseline thresholds.

Comparing results between 'main' and 'pull_request'

Host '373323ac6ad6' with 4 'x86_64' processors with 15 GB memory, running:
#26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024

MyBenchmark

lipsum metrics

Time (wall clock): results within specified thresholds, fold down for details.

| Time (wall clock) (ns) * | p0 | p25 | p50 | p75 | p90 | p99 | p100 | Samples | |:----------------------------------------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:| | main | 7980 | 8036 | 8221 | 8360 | 8544 | 9118 | 9929 | 100 | | pull_request | 7979 | 8098 | 8298 | 8471 | 8684 | 9527 | 9839 | 100 | | Δ | -1 | 62 | 77 | 111 | 140 | 409 | -90 | 0 | | Improvement % | 0 | -1 | -1 | -1 | -2 | -4 | 1 | 0 |

lipsum-zh metrics

Time (wall clock): results within specified thresholds, fold down for details.

| Time (wall clock) (ns) * | p0 | p25 | p50 | p75 | p90 | p99 | p100 | Samples | |:----------------------------------------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:| | main | 1473 | 1479 | 1488 | 1524 | 2033 | 2615 | 2648 | 100 | | pull_request | 1515 | 1547 | 1567 | 1620 | 1664 | 1856 | 1856 | 100 | | Δ | 42 | 68 | 79 | 96 | -369 | -759 | -792 | 0 | | Improvement % | -3 | -5 | -5 | -6 | 18 | 29 | 30 | 0 |

medium-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

| Time (wall clock) (μs) * | p0 | p25 | p50 | p75 | p90 | p99 | p100 | Samples | |:----------------------------------------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:| | main | 41 | 41 | 41 | 41 | 42 | 44 | 45 | 100 | | pull_request | 41 | 41 | 42 | 43 | 43 | 48 | 48 | 100 | | Δ | 0 | 0 | 1 | 2 | 1 | 4 | 3 | 0 | | Improvement % | 0 | 0 | -2 | -5 | -2 | -9 | -7 | 0 |

small-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

| Time (wall clock) (ns) * | p0 | p25 | p50 | p75 | p90 | p99 | p100 | Samples | |:----------------------------------------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:| | main | 4489 | 4506 | 4514 | 4530 | 4702 | 5837 | 5883 | 100 | | pull_request | 4529 | 4592 | 4743 | 4792 | 4858 | 5071 | 5072 | 100 | | Δ | 40 | 86 | 229 | 262 | 156 | -766 | -811 | 0 | | Improvement % | -1 | -2 | -5 | -6 | -3 | 13 | 14 | 0 |

strong metrics

Time (wall clock): results within specified thresholds, fold down for details.

| Time (wall clock) (μs) * | p0 | p25 | p50 | p75 | p90 | p99 | p100 | Samples | |:----------------------------------------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:| | main | 20 | 20 | 20 | 20 | 20 | 22 | 24 | 100 | | pull_request | 20 | 20 | 21 | 21 | 21 | 22 | 23 | 100 | | Δ | 0 | 0 | 1 | 1 | 1 | 0 | -1 | 0 | | Improvement % | 0 | 0 | -5 | -5 | -5 | 0 | 4 | 0 |

tiny-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

| Time (wall clock) (ns) * | p0 | p25 | p50 | p75 | p90 | p99 | p100 | Samples | |:----------------------------------------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:|--------:| | main | 441 | 445 | 448 | 450 | 455 | 469 | 487 | 100 | | pull_request | 465 | 471 | 479 | 482 | 493 | 504 | 543 | 100 | | Δ | 24 | 26 | 31 | 32 | 38 | 35 | 56 | 0 | | Improvement % | -5 | -6 | -7 | -7 | -8 | -7 | -11 | 0 |