How best to deal with buffered entries?

[tunetheweb]

I'm trying to implemented support of this into web-vital.js.

For future events this works well. For example, for LCP events I can do the following:

• Set up LCP PerformanceObserver
• Set up SoftNav PerformanceObserver
• As each new Soft Nav comes in, I can "finalize" the previous LCP metric and emit it, and prepare to gather the next LCP entries as they come in.
• As each LCP entry comes in, I can update the current highest LCP metric ready for it to be finalised too (either by a new SoftNav, a click, or a backgrounding of the window - the last one is not strictly accurate btw but the best we can do until we get the Pending Beacon API).

I presume this will work as the events should be queued in order, so a SoftNav should always be emitted before it's first LCP entry and testing appears to show that is the case. So my first question is: Is that a valid assumption?

However, I'm running into problems when processing buffered events that happened before the library was initiated. The flow is basically the following:

• Set of LCP PerformanceObserver. For LCP entries for SoftNavs that occured before the library was initialised I will get some buffered events, but don't have the equivalent SoftNav details yet to get the SoftNav start time, to calculate the actual LCP time. So I need to store these buffered events to process later. This will increase memory usage of the library (and I am also concerned that if I miss a SoftNav event, then this might never be cleared down).
• Set up SoftNav PerformanceObserver. I will get some buffered events here too. If I do this first and don't have the LCP entries, then can't process them either, as I need to finalise each LCP entry based on a new SoftNav appearing and wouldn't have those LCP entries yet.
• If I have both LCP PO, and SoftNav PO events, then I can process them, but I need to wait until I have both, and then need to process the LCP PO events in order, referencing the given Soft Nav entries, before then entering the "live" mode where I can just monitor the latest incoming events (SoftNav or LCP) and treat appropriately as per the first case above.

In effect, I need to synchronise the results of two different PerformanceObservers and ensure that no entries get dropped, and that no entries outstay their welcome. It's possible, but proving a bit tricky...

I thought of setting one PerformanceObserver with both LCP and SoftNav entries to allow buffered ones to be replayed in order, using entryTypes but that doesn't support the buffered flag otherwise that could work quite well and only require processing one LCP at a time (as we do currently). But, given the lack of buffered support that is not an option. 😔

While we're on the subject, I have an additional problem that we don't "finalize" an LCP event, until a SoftNav occurs, at which point the URL has changed. As users will want to associate the LCP event with the previous URL to send to their analytics provider, that needs to be available too. We do need to get the SoftNav entry anyway to calculate the correct LCP start time so could store the URL form that and add the URL to the metric emitted by the library, but for the initial navigation and initial LCP we don't have that URL so need to set up ANOTHER PerformanceObserver to get the initial navigation event (which is PerformanceNavigationTiming rather than a SoftNavigationEntry so again slightly different). Which makes the problem even trickier as now dealing with three PerformanceObservers... Multiple that up by the 6 metrics that we track (all of which are basically currently handled independently at the moment) and it gets even more complicated.

This all would be considerably simplified, if the LCP event included the Soft Navigation startTime (or the LCP time was based on this rather than on navigationStart) and also if it included the navigation URL it was the LCP for. Or, alternatively, if it included a reference to the navigation entry (full or soft as appropriate) that the LCP was emitted for, so I can get those two bits of information more easily. But, again, that's not currently possible AFAIK.

I'm sure I've over complicated this, so any thoughts on a better way of doing this?

[mmocny]

I thought of setting one PerformanceObserver with both LCP and SoftNav entries

FYI you can call .observe() multiple times on a single PO and achieve this using buffered flag:

let entries = [];
let obs = new PerformanceObserver(list => entries.push(...list.getEntries()));
['paint','event','largest-contentful-paint','element','resource','navigation'].forEach(type => obs.observe({ type, buffered:true, durationThreshold: 0 }));

I also checked and it looks like entries are sorted so I think you'll get them in expected order-- though I'm not sure if thats guarenteed.

[tunetheweb]

There's an easier solution. You CAN look up the navigation entries directly with:

• performance.getEntries("navigation")[X] - where X is the NavigationId - 1 (basically 0 for the primary nav, which I think is the only one that should be used?).
• performance.getEntries("soft-navigation")[X] - where X is the NavigationId - 2

So that massively reduces the complexity here. Completely forgot about that API.

Not everything is in that API, but the navigation ones are, and those are the only ones I need.