TimGuiteDiamond
commented
4 years ago The Profiler component has indeed proven successful and useful.
It is amuch easier way of extracting the information we are interested in and should allow us to perform useful comparisons on branch changes in the future. Although the method I lay out below is manual, it has the potential to be automated much more in the future.
Verifying previous results
We know from our previous measurements that the master branch is currently quite good at rendering many readbacks which are all looking at a single PV, and that without throttling, performance takes a big hit when looking at many different PVs.
To verify the usefulness of the Profiler component, I performed the same experiments again, measuring 3 statistics available through the onRender callback:
- actualDuration - time spent rendering the committed update
- baseDuration - estimated worst case cost of rendering the subtree
- reconciliationTime - commitTime - startTime (called React Tree Reconciliation in the Chrome Devtools performance view)
Methodology
I initially set up the Profiler component around the top level DynamicPage widget, which holds the performance page and all of the Readbacks when they are shown. For our master branch, a simple use of the onRender callback to provide comma separated outputs sufficed. I tested the performance page with various numbers of Readbacks, all looking at sim://ramp#00.
Once I began testing Readbacks looking at different PVs though, a measurement issue arose. From previous investigations, it was seen that without throttling, many reconciliations were being performed. These reconciliations are obvious to the developer as being related but the Profiler component does not have this knowledge. Therefore, all timings within a window of half the simulation trigger were aggregated. This is not ideal, as it does not account for the gaps in time between reconciliations, which could be a cause of visible delay for the user. Nevertheless, it seems to function relatively well. A more sophisticated system could be used in the future.
Again, various numbers of Readbacks were tested, now looking at their own PVs (sim://ramp#00, sim://ramp#01 etc). I then merged the performance branch with our simple throttling implementation and repeated the experiment.
Results
Here are the graphs showing how actualDuration, baseDuration and Reconciliation Time changed with widget number:
Actual Duration
Base Duration
Reconciliation Time
Analysis
Interestingly, the actual duration does not seem to change much between the three tests. I understand this to mean that once React has performed its reconciliation, there is not a significant difference between the three in the time taken to write this to the real DOM. This would make sense as they are painting the same number of widgets and making very similar changes to the DOM.
The base duration shows a clear increase for the test without throttling, showing several orders of magnitude change by 100 widgets. This represents the worst-case render time; without throttling, React would potentially have to check the entire subtree for each new value update. So this is expected and useful information. As expected, the master and throttled versions are very similar as they only perform the reconciliation once.
Reconciliation time similarly shows the expected difference between the tests, although it is perhaps more representative of the delays which would be seen by the user than base duration. We can see clearly here that even our simple throttling tool performs well compared to the case where all readbacks are looking at the same PV.
Caveats
For the throttled case, renders are sometime split across multiple reconciliations, depending on how quickly the store was updated and how the updates interacted with the throttling periods.
These times were aggregated in the Profiler function but are clearly visible to the user. The gaps between reconciliations are not properly accounted for in these measurements. This is an effect of how we have set up the tests, throttling and simulation. For a more direct comparison between the master and throttled branch, we should try to trigger all simulation updates at once, rather spreading it across many function calls. Then we would expect to see very similar performance metrics between master and throttled, both from the profiler and in Chrome Devtools.
Future Work
This represents a big step in the right direction with regards to making it easier to measure the impact of our changes on performance. We should attempt to automate this further and investigate how we can integrate this with our CI pipeline, to provide some simple statistics at the least.
Further work could be done to see how different widgets, e.g ProgressBar, compare with the results we have shown here.
aawdls
We have made some good headway regarding performance and measuring this at different levels.
However, the current methodology is manual and time consuming. It would be great to have an automated measurement tool of some kind. Ideally this would integrate with our CI platform to make it obvious to us when we have introduced significant performance slowdown into the application.
As a first step it would be useful to have a tool which measures renders at various levels and outputs this data in a useful format, to speed up the process of measuring multiple renders and extracting the render time, reconciliation time etc.
React Profiler looks like a useful tool and this article describes its use and benefit, as well as linking to a live project on Github which is using this profiling mechanism.