callstack / reassure

Performance testing companion for React and React Native
https://callstack.github.io/reassure/
MIT License
1.21k stars 25 forks source link

feat: expose more in markdown details #503

Open mdjastrzebski opened 1 month ago

mdjastrzebski commented 1 month ago

Summary

Test plan

changeset-bot[bot] commented 1 month ago

🦋 Changeset detected

Latest commit: d6676dde43155482b7625b3ecfe41bf3a79cfcd6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages | Name | Type | | --------------------------- | ----- | | reassure | Minor | | @callstack/reassure-compare | Minor | | @callstack/reassure-cli | Minor | | @callstack/reassure-danger | Minor | | @callstack/reassure-logger | Minor | | @callstack/reassure-measure | Minor |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

github-actions[bot] commented 1 month ago

Performance Comparison Report

Significant Changes To Duration

Name Type Duration Count
fib 32 function 101.7 ms → 116.6 ms (+14.9 ms, +14.7%) 🔴 1 → 1
fib 31 function 62.9 ms → 72.0 ms (+9.1 ms, +14.4%) 🔴 1 → 1
fib 30 function 39.1 ms → 44.7 ms (+5.6 ms, +14.4%) 🔴 1 → 1
Show details | Name | Type | Duration | Count | | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | fib 32 | function | **Baseline**
Mean: 101.7 ms
Stdev: 2.4 ms (2.3%)
Runs: 101.2 103.6 102.1 102.2 95.1 102.0 102.8 102.2 102.5 102.9
Warmup runs: 101.1

**Current**
Mean: 116.6 ms
Stdev: 0.3 ms (0.2%)
Runs: 116.4 116.7 116.5 116.3 116.6 116.6 116.4 116.5 117.2 116.5
Warmup runs: 116.1 | **Baseline**
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1

**Current**
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1 | | fib 31 | function | **Baseline**
Mean: 62.9 ms
Stdev: 0.1 ms (0.1%)
Runs: 62.9 62.9 63.0 62.8 62.8 62.9 62.8 62.9 62.9 63.1
Warmup runs: 62.5

**Current**
Mean: 72.0 ms
Stdev: 0.3 ms (0.4%)
Runs: 71.9 72.0 71.7 72.0 72.1 72.6 71.6 71.9 71.6 72.1
Warmup runs: 71.0 | **Baseline**
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1

**Current**
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1 | | fib 30 | function | **Baseline**
Mean: 39.1 ms
Stdev: 0.3 ms (0.7%)
Runs: 38.5 38.8 39.4 39.3 39.2 38.9 38.9 39.2 39.4 39.1
Warmup runs: 39.1

**Current**
Mean: 44.7 ms
Stdev: 0.9 ms (2.0%)
Runs: 44.5 44.9 47.1 44.2 44.3 44.4 44.2 44.7 44.4 44.3
Warmup runs: 43.9 | **Baseline**
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1

**Current**
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1 |

Meaningless Changes To Duration

Show entries | Name | Type | Duration | Count | | ---------------------------------- | ------ | ------------------------------------- | ------ | | Async Component | render | 104.3 ms → 105.1 ms (+0.8 ms, +0.8%) | 7 → 7 | | InitialRenders 1 | render | 0.8 ms → 0.9 ms (+0.1 ms, +12.5%) | 2 → 2 | | InitialRenders 3 | render | 0.9 ms → 1.0 ms (+0.1 ms, +11.1%) | 4 → 4 | | ManyRenderIssues | render | 2.1 ms → 1.9 ms (-0.2 ms, -9.5%) | 5 → 5 | | Other Component 10 | render | 65.4 ms → 68.5 ms (+3.1 ms, +4.7%) | 4 → 4 | | Other Component 10 legacy scenario | render | 65.5 ms → 64.9 ms (-0.6 ms, -0.9%) | 4 → 4 | | Other Component 20 | render | 68.3 ms → 66.6 ms (-1.7 ms, -2.4%) | 4 → 4 | | RedundantUpdates | render | 0.8 ms → 0.8 ms | 2 → 2 |
Show details | Name | Type | Duration | Count | | ---------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Async Component | render | **Baseline**
Mean: 104.3 ms
Stdev: 4.6 ms (4.4%)
Runs: 108 105 109 105 96 103 108 108 104 97
Warmup runs: 99

**Current**
Mean: 105.1 ms
Stdev: 4.9 ms (4.6%)
Runs: 106 98 106 106 112 112 108 99 101 103
Warmup runs: 105 | **Baseline**
Mean: 7
Stdev: 0 (0.0%)
Runs: 7 7 7 7 7 7 7 7 7 7
Render issues:
- Initial updates: 1

**Current**
Mean: 7
Stdev: 0 (0.0%)
Runs: 7 7 7 7 7 7 7 7 7 7
Render issues:
- Initial updates: 1 | | InitialRenders 1 | render | **Baseline**
Mean: 0.8 ms
Stdev: 0.4 ms (52.7%)
Runs: 1 1 1 1 1 1 1 0 0 1
Warmup runs: 3

**Current**
Mean: 0.9 ms
Stdev: 0.3 ms (35.1%)
Runs: 1 1 1 1 1 0 1 1 1 1
Warmup runs: 2 | **Baseline**
Mean: 2
Stdev: 0 (0.0%)
Runs: 2 2 2 2 2 2 2 2 2 2
Render issues:
- Initial updates: 1

**Current**
Mean: 2
Stdev: 0 (0.0%)
Runs: 2 2 2 2 2 2 2 2 2 2
Render issues:
- Initial updates: 1 | | InitialRenders 3 | render | **Baseline**
Mean: 0.9 ms
Stdev: 0.7 ms (82.0%)
Runs: 0 2 1 1 2 1 1 0 1 0
Warmup runs: 2

**Current**
Mean: 1.0 ms
Stdev: 0.8 ms (81.6%)
Runs: 0 0 1 0 2 2 1 1 2 1
Warmup runs: 2 | **Baseline**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 3

**Current**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 3 | | ManyRenderIssues | render | **Baseline**
Mean: 2.1 ms
Stdev: 0.6 ms (27.0%)
Runs: 3 2 2 2 1 2 2 3 2 2
Warmup runs: 1

**Current**
Mean: 1.9 ms
Stdev: 0.6 ms (29.9%)
Runs: 2 2 2 2 2 3 2 1 1 2
Warmup runs: 1 | **Baseline**
Mean: 5
Stdev: 0 (0.0%)
Runs: 5 5 5 5 5 5 5 5 5 5
Render issues:
- Initial updates: 2
- Redundant updates: 2 (3, 4)

**Current**
Mean: 5
Stdev: 0 (0.0%)
Runs: 5 5 5 5 5 5 5 5 5 5
Render issues:
- Initial updates: 2
- Redundant updates: 2 (3, 4) | | Other Component 10 | render | **Baseline**
Mean: 65.4 ms
Stdev: 3.2 ms (5.0%)
Runs: 60 68 68 63 67 70 64 62 68 64
Warmup runs: 199

**Current**
Mean: 68.5 ms
Stdev: 3.5 ms (5.1%)
Runs: 63 68 69 66 64 69 70 69 73 74
Warmup runs: 80 | **Baseline**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 1

**Current**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 1 | | Other Component 10 legacy scenario | render | **Baseline**
Mean: 65.5 ms
Stdev: 3.7 ms (5.7%)
Runs: 70 63 69 64 64 66 68 67 57 67
Warmup runs: 61

**Current**
Mean: 64.9 ms
Stdev: 4.0 ms (6.1%)
Runs: 61 64 63 61 65 62 64 67 68 74
Warmup runs: 68 | **Baseline**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 1

**Current**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 1 | | Other Component 20 | render | **Baseline**
Mean: 68.3 ms
Stdev: 6.0 ms (8.9%)
Runs: 65 63 71 65 71 64 60 55 65 79 70 68 72 68 79 72 77 68 64 69
Warmup runs: 65

**Current**
Mean: 66.6 ms
Stdev: 4.1 ms (6.1%)
Runs: 63 58 66 66 70 63 65 67 65 71 68 67 75 66 74 66 64 61 67 70
Warmup runs: 64 | **Baseline**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 1

**Current**
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Render issues:
- Initial updates: 1 | | RedundantUpdates | render | **Baseline**
Mean: 0.8 ms
Stdev: 0.8 ms (98.6%)
Runs: 0 0 1 1 2 1 2 1 0 0
Warmup runs: 1

**Current**
Mean: 0.8 ms
Stdev: 0.9 ms (114.9%)
Runs: 0 2 1 1 2 0 0 0 2 0
Warmup runs: 1 | **Baseline**
Mean: 2
Stdev: 0 (0.0%)
Runs: 2 2 2 2 2 2 2 2 2 2
Render issues:
- Redundant updates: 1 (1)

**Current**
Mean: 2
Stdev: 0 (0.0%)
Runs: 2 2 2 2 2 2 2 2 2 2
Render issues:
- Redundant updates: 1 (1) |

Render Count Changes

There are no entries

Render Issues

Name Initial Updates Redundant Updates
Other Component 10 1 🔴 -
Other Component 10 legacy scenario 1 🔴 -
Other Component 20 1 🔴 -
InitialRenders 1 1 🔴 -
InitialRenders 3 3 🔴 -
RedundantUpdates - 1 (1) 🔴
ManyRenderIssues 2 🔴 2 (3, 4) 🔴
Async Component 1 🔴 -

Added Scenarios

There are no entries

Removed Scenarios

There are no entries

Generated by :no_entry_sign: dangerJS against d6676dde43155482b7625b3ecfe41bf3a79cfcd6