maxyinger commented 2 years ago

Performance Report April 2022

This is a performance report covering progress in the editor from prior to performance work starting to the current state of editor performance. This is comparing branches main at hash 1d5c7228e to branch perf-sidebar from latest perf PR

Environment

Interaction tests run on a fully specced 2018 13" Macbook Pro on this test story which consists of 1 page with 50 square shape elements, each with a bounce animation, media pane open and layers panel expanded.

Each test interaction will document the main thread of 4 different builds:

1d5c7228e React development build
1d5c7228e React production build
perf-sidebar React development build
perf-sidebar React production build

Development builds are used to compare progress with last performance report #10158, whereas production builds are used to document approximate user experience progress. We are omitting the React flame graphs for now since the main focus here is user facing performance differences rather than how the interactions became more performant internally. Happy to document and add in React flame graphs adhoc if requested though.

Branch: 1d5c7228e	Branch: perf-sidebar

TODO

[x] update selection interaction tests
[x] drag to update element position tests (drag start & end)
[x] reorder element layer interaction test
[x] ~possible performance documentation on canvas generation cache improvements~ this is hard to clearly document, but with the new setup, we are hitting the cache more which should save us some ~1s calls to convert a story page to a canvs

TLDR

The most applicable metric to gauge user interaction is the time the interaction took in the react production build, because that's the build shipped to our users. Here's a small table highlighting the current performance increases for those interactions. Each interaction is elaborated on below with comments in this thread.

Make note that we are officially under 100ms for all of these which is the bar Google sets for the Interaction section of the RAIL model:

0 - 100ms Respond to user actions within this time window and users feel like the result is immediate. Any longer, and the connection between action and reaction is broken.

Interaction	Old Production Time	Current Production Time	Rough Performance Increase	Currently Under 100ms
Update Selection	110ms	35ms	3.1x	✅
Update Element Position - Drag Start	50ms	30ms	1.7x	✅
Update Element Position - Drag End	195ms	95ms	2.0x	✅
Reorder Layer Position	120ms	30ms	4.0x	✅

Fixes #11120

maxyinger commented 2 years ago

Selection Update

Recording of Interaction https://user-images.githubusercontent.com/35983235/163421161-d03134d9-8ce1-4f77-b393-da6e59f73cab.mp4

Note This video was taken on the perf-sidebar branch with react in the development build. Recorded the video on a 16" macbook pro m1 which is why the performance appears to be faster than the perf-sidebar dev build screenshot below (which was recorded on a 13" macbook pro with intel chip)

React Development Build

1d5c7228e	perf-sidebar
~320ms	~70ms

React Production Build

1d5c7228e	perf-sidebar
~110ms	~35ms

maxyinger commented 2 years ago

Drag To Reposition Element

Recording of Interaction

https://user-images.githubusercontent.com/35983235/163423716-329e056e-9b1a-4749-b90b-5c10387f72bc.mp4

Note This video was taken on the perf-sidebar branch with react in the development build. Recorded the video on a 16" macbook pro m1 which is why the performance appears to be faster than the perf-sidebar dev build screenshot below (which was recorded on a 13" macbook pro with intel chip)

React Development Build

Drag Start	1d5c7228e	perf-sidebar
~150ms	~55ms

Drag End	1d5c7228e	perf-sidebar
~585ms	~140ms

React Production Build

Drag Start	1d5c7228e	perf-sidebar
~50ms	~30ms

Drag End	1d5c7228e	perf-sidebar
~195ms	~95ms

maxyinger commented 2 years ago

Reorder Element Layer

Recording of Interaction https://user-images.githubusercontent.com/35983235/163426028-56d1e544-1710-46e1-a9f6-9e783b6aa84c.mp4

Note This video was taken on the perf-sidebar branch with react in the development build. Recorded the video on a 16" macbook pro m1 which is why the performance appears to be faster than the perf-sidebar dev build screenshot below (which was recorded on a 13" macbook pro with intel chip)