search

akamai / boomerang

End user oriented web performance testing and beaconing
http://akamai.github.io/boomerang/
Other
1.86k stars 292 forks source link

Confused why TTI was marked when page was still "busy" #307

Closed SeanWhipple closed 3 years ago

SeanWhipple commented 4 years ago

I have a datapoint that came out of boomerang where I don't quite understand why CTCI was marked the way it was. The data I received from Boomerang is below

tti: 2586
visually ready: 783

frame rate timeline: 0266671 => [2, 6, 6, 6, 7, 1]
page busy timeline: 200500000__ => [0, 50, 0, 0, 100]
delayed interaction timeline: 0*5*01 => [0, 0, 0, 0, 0, 1]

epoch time: kepzx61z
frame start time: kepzx84l

The last two time stamps give a startTime of 2686 ms, which is 100 ms over when TTI was marked. To me this suggests that when Boomerang began monitoring the page Boomerang already considered the page "interactive". What I don't understand is why...

If the page was interactive the page would have needed to be visually ready, which it was. The page would also need to have 5 intervals where the frame rate was sufficiently high, no delayed interactions, and the page wasn't busy. The first five intervals for the frame rate and delayed interaction timelines both met that criteria. But the page busy timeline suggests that the 5th interval was NOT interactive. So I don't understand why it was marked as such

Is it possible there is a race condition...a situation where the code to calculate if TTI is ready is run but the Page Busy data is not all there? TTI gets marked but then the Page Busy timeline gets updated after?

nicjansma commented 3 years ago

@SeanWhipple yeah after reviewing the code, I think your theory is correct -- there is a potential race condition where if the "busy" log hasn't been set for a bucket yet, the TTI calculation may think it's a non-busy period.

The "busy" monitor runs every 100ms and will estimate the busyness for the previous 100ms when it executes.

The TTI calculator could run at a couple different points, either right after Page Load is done or at a 1,000ms interval afterwards (depending on the configuration).

It's possible with these two timers/events that the TTI calculator runs "between buckets 4 and 5", and thinks everything is rosy during bucket=5 so stamps TTI as having been at bucket=0. Then the data gets filled in for bucket 5 and the calculation is wrong.

Do you have a sense of how often this happens with your data? Did you stumble upon this or can you find it with other patterns in your data?

SeanWhipple commented 3 years ago

Unfortunately this was something I just stumbled upon. I don't have good data to say how often this is happening. Just saw a data point and was hoping to get clarity

nicjansma commented 3 years ago

Ok, let me think a bit about how we could address this, might be affected by https://github.com/akamai/boomerang/issues/315 too

nicjansma commented 3 years ago

Hopefully addressed in https://github.com/akamai/boomerang/pull/326

scottgifford commented 3 years ago

(colleague of Sean's here)

To answer your question on how often this happens Nic, in our last 7 days of data we have about 292.4M data points, and for 0.6M of them we could not calculate TTI based on the timeline data boomerang recorded (about 0.2%). Additionally there are about 60K where our re-calculation is different from the recorded TTI by more than 100ms.

Eager to see if the fix in #326 addresses this, we'll try to get this deployed in the coming weeks!