Open bgirard opened 7 months ago
Regarding track
, here's a simple strawman:
In Chromium:
name
, startTime
, and endTime
to define the track name of marks/measures, but we could use this options.detail.track
value if provided.(We may also want to change the default hash strategy to merge more things...)
Regarding the concept of optional slow/heavy categories:
track
helps here.TRACE events already support custom track by name.
If you mean Perfetto (trace-viewer too?) yes but not tools like DevTools, the primary frontend for performance.measure
for most web product developers, there's no way to separate events into track in the frontend and you end up with something unmanageable like in my linked discussion example from a few years ago.
here's a simple strawman
That would be great. Suggesting a options.detail.track
string as the standard for product and tools to communicate this information. Or perhaps options.detail.track.name
to be more future proof if we ever needed to specify more track information in the future.
By glancing over at the Perfetto SDK/API, the one feature we would want would be a way to name the track. But that's not essential.
If you mean Perfetto (trace-viewer too?) yes but not tools like DevTools, the primary frontend for
performance.measure
for most web product developers, there's no way to separate events into track in the frontend and you end up with something unmanageable like in my linked discussion example from a few years ago.
Fair enough! I think that becomes a distinct feature request for the DevTools product.
That would be great. Suggesting a
options.detail.track
string as the standard for product and tools to communicate this information. Or perhapsoptions.detail.track.name
to be more future proof if we ever needed to specify more track information in the future.
+1
By glancing over at the Perfetto SDK/API, the one feature we would want would be a way to name the track. But that's not essential.
I know that capability is supported by Perfetto UI, but I guess it isn't "exposed"?
I know that capability is supported by Perfetto UI, but I guess it isn't "exposed"?
Yes, it's supported. Here's an example of what I have so far with a private implementation of this proposal in React Native (showing the tracks coming from UserTimings, not shown are the existing native tracks). There's a good chance that these improvement will be merged within open source React Native eventually and fixing this spec issue would help:
There's shared JavaScript code running both on Web and React Native. I'm moving all the performance logging code away from custom logger to use UserTimings with this custom track feature. React Native then creates custom tracks using the Perfetto SDK and ideally someday Chrome/Firefox Devtools will implement this feature as well.
Jumping in a little late to the conversation but folks might find this interesting.
This is an experimental API we recently made available in Chrome DevTools for extending the Performance panel. It basically fulfills the request for customizing the track User Timings appear within in the timeline: https://bit.ly/rpp-e11y.
It relies on the detail
field of User Timings as the payload container and specifies a format for the data that allows to customize how the timing is displayed, including track, color and details info.
Extension data is expected to be contained under detail.devtools
. For example, if you wanted to add data under a track called "React Profiler", you could do so by making a call similar to:
const measureOptions = {
// ...
detail: {
devtools: {
metadata: {
dataType:"track-entry",
extensionName: 'React extension',
},
color: 'primary',
// Track the entry will be displayed in
track: 'React Profiler',
}
},
};
performance.measure( "A measurement", measureOptions);
The API is experimental because we are still looking for feedback from the ecosystem to launch something that's valuable, any thoughts would be appreciated!
That sounds promising. I'll give it a shot in the next 2 weeks and will share some feedback.
Found some time to test it. At first it didn't work after enabling the flag but I tried it in Canary and it worked there. I was able to easily retrofit Facebook Web's performance measures to this suggested API and get the following view:
There's more useful logging, like I showed in Perfetto, that we would expose now that we have the concept of tracks. Having this API standardized would really help and React Native would likely follow suit as well.
My spec relevant feedback is:
Non spec relevant feedback:
Thank you so much for taking the time to test the API and come back with such detailed feedback!
Minimal overhead should be the primary concern.
One way we thought about avoiding overhead in hot paths was by calling the performance.measure API with the measurements after the fact (since it can be tricky to ensure the call to the API is quick) . Say for example measurements are taken efficiently inside React component rendering, but the calls to the API are added at the end of the recording with their original timestamps, which UserTimings L3 allows to do. We are still figuring out what's the most ergonomic and maintainable way to achieve this, especially because the calls to the API might happen after trace events stop being buffered when the recording stops, but it's technically possible. How does this sound to you?
Color should be optional.
That's a good point and I agree. Will make sure we consider this.
detailsPairs is a nice API as well. It would be nice getting an API to inject a custom component there
This is something we have been thinking of but aren't sure what shape will be the best for it. We are waiting to get more feedback of possible use cases to have a better idea of this, but I think it isn't a blocker since it could be added progressively .
We could consider adding an API for the page to list what advanced category it supports and letting the devtool toggle them.
To make sure I'm following, would this work as a "filter" to allow the developer to select what will be recorded from an extension? If so, that could be pretty interesting. Will investigate if this is also beneficial for other potential early adopters.
I was expecting to see this as a sub-view of Timings. Even just ordering it higher in the view near Timings would be nice. Otherwise it can get missed easily.
That's also a fair point. Will also make sure we consider moving extension tracks closer to timings and likely above the main thread. Also note that you can customize the track order in the Performance panel (right click on the track name -> configure tracks)
Happy to provide more feedback 1:1
Sounds good! I appreciate that. I'll definitely set up something if the conversation extends and needs more space!
calling the performance.measure API with the measurements after the fact
That moves a lot of complexity on to user code, regresses app size to implement a buffering mechanism. It would be much better for UserTimings to buffer itself if it needs to defer something costly. Web binding is cheap enough on all platform. So it's really just about not having too much redundant data in the object, or being able to easily reuse references between calls.
but the calls to the API are added at the end of the recording
That's the other problem. There's no way to observe that from web APIs currently. There's no 'flush your performance entries now' API. And as long as we can solve the problem above, I think we should continue to punt on adding one.
To make sure I'm following, would this work as a "filter" to allow the developer to select what will be recorded from an extension?
Yes exactly. It's common to want to select which 'categories' you'll be logging and paying for the cost for.
After looking into the options to natively work around the performance overhead of the performance
API, we are also exploring using the console
API instead of performance
. In particular, because performance
(including measure
/mark
) is meant to buffer user observable data, which contributes to the API's slowness and is difficult to bypass natively because of the semantics of the API itself. console
on the other hand is meant specifically to interact with devtools. For example, the sole purpose of console.timeStamp
is to add data to the DevTools Performance panel, so it can be more easily optimized natively for performance, especially in the no-op cases (when not profiling / tracing with the perf panel).
We've added an exploration for this in the whatwgh repo: https://github.com/whatwg/console/issues/140
The main disadvantage of this approach is the disregard for the existing adoption of user timings in the ecosystem, since people are using it for adding stuff to the performance tool but also to power telemetry / logs / etc. An alternative would be to enable performance.measure
/mark
instrumentation only on profiling/dev builds of apps but still the performance of the profiled application itself would be affected, potentially leading to misleading recordings.
meant to buffer user observable data, which contributes to the API's slowness and is difficult to bypass natively because of the semantics of the API itself.
I agree. In React Native's implementation this is adding significant overhead. This is being optimized in https://github.com/facebook/react-native/issues/45122. But I agree that fundamentally the semantics of the API are going to get in the way here.
Personally I don't feel strongly if it should go under either console
or performance
. The slight problem with putting it under console
is that the data should be going under the performance
tab of DevTools, not the console
tab. So that also feels smelly.
Ultimately I think either approach has pros and cons. I think having cross vendor alignment here is really the most important. I would personally support either variant. If we go with measure
we need to either have a fast implementation or add an option to avoid creating `performanceEntry's.
I touched on this need in this discussion many years ago: https://github.com/w3c/user-timing/issues/68#issuecomment-555266453 .
Frankly I've ran into this many times. You can see some screenshots in that discussion for how heavily user timings are used at Meta on web. More recently I'm looking at a tooling integration where this would be useful again. React-native already support user-timings but I'd like to forward it to the system's Perfetto on certain platforms and builds. Tracks are an important feature of Perfetto because it organizes the events in the Perfetto frontend.
Ideally I'd like to see the following happen here:
mark
andmeasure
. We document the expectation that it helps categories events into tracks in the UI to organize them together. This would all be non-normative expectations of downstream tools of course.