[Bug] WebVitals (e.g. CLS) being shown for non-Chromium browsers

elastic / apm-agent-rum-js

https://www.elastic.co/guide/en/apm/agent/rum-js/current/index.html

MIT License

279 stars 134 forks source link

[Bug] WebVitals (e.g. CLS) being shown for non-Chromium browsers #928

Closed paulb-elastic closed 3 years ago

paulb-elastic commented 3 years ago

It has been observed that WebVital values are being recorded from browsers that do not support WebVitals. For example, we can see that there is a CLS score for a Safari on iPhone browser:

[ ] Ensure docs mention a recommendation to upgrade to the latest RUM agent, including the compatibility matrix (https://github.com/elastic/observability-docs/issues)

vigneshshanmugam commented 3 years ago

I have verified with a test page on all safari versions from 12 - 14 on Mac OSX and was not able to reproduce the bug. Test was done through Saucelabs.

Safari even on tech preview does not support CLS metrics. So this is highly likely a bug in APM server user agent parser or it could from users who have extensions installed that alters the user agent header locally.

/cc @elastic/apm-server Team are you aware of any bugs in the the user agent parser that mistakes user agent header for Chrome as Safari in OSX?

axw commented 3 years ago

What's highlighted there is the original, unparsed User-Agent header value. So I don't see how it could be a case of the server mistaking the user agent for Chrome.

Which version of APM Server is being used? One possibility is that data is getting mixed up between events. I have had no reason to believe this is happening previously, but there has been some upheaval in the decoding logic recently.

simitt commented 3 years ago

@paulb-elastic with which APM Server version did you observe this behavior?

paulb-elastic commented 3 years ago

@cachedout are you able to confirm what version of the APM Server the prod elastic.co data is being sent to, that ends up in the dev-next cluster? cc @simitt

simitt commented 3 years ago

I already received the information about the cluster in the mean time, thank you.

cachedout commented 3 years ago

@cachedout are you able to confirm what version of the APM Server the prod elastic.co data is being sent to, that ends up in the dev-next cluster?

Traffic is mirroed to each of the APM servers associated with the clusters in question. So, for dev-next, the latest snapshot is deployed daily.

paulb-elastic commented 3 years ago

Through a lot of investigation, it looks like these are coming from Chromium based clients that are changing the UA string, which is why we categorise them as another browser (e.g. Safari on iPhone), but the CLS is being able to be determined because it is actually a Chromium based browser. As such, closing this issue.