mozilla / bedrock

Making mozilla.org awesome, one pebble at a time
https://www.mozilla.org
Mozilla Public License 2.0
1.17k stars 913 forks source link

Investigate alternate / 1st party analytics #7083

Closed alexgibson closed 1 year ago

alexgibson commented 5 years ago

Description

With the rise of adblockers and tracking protection, we should investigate a 1st party, self-hosted, analytics solution. This will become especially useful for user agents that default to privacy always on.

Example: Firefox Nightly with a new profile visiting Mozorg:

Screenshot 2019-04-16 at 18 10 14
alexgibson commented 5 years ago

/cc @hoosteeno

hoosteeno commented 5 years ago

The code landing in a future version of Firefox will block cookies from certain third party providers identified by Disconnect.me. GA's basic analytics cookies (including GA and GTM) will not be blocked.

Analysis here.

pmac commented 5 years ago

I've tested the site in a fresh profile in Nightly 70.0a1 and it does indeed block GTM loading. I'm not sure about cookies, but it doesn't look like we'll be getting any data from Firefox users once 70 is released.

hoosteeno commented 5 years ago

A fresh Nightly profile on today's version on standard ETP is not blocking GTM, nor is it highlighting the presence of trackers in the shield icon (See attached). GA will work as expected under these conditions.

Screen Shot 2019-08-28 at 2 10 36 PM

In standard mode, clicking on the shield icon reveals a UI that explains what was allowed: google-analytics.com was allowed.

Visiting about:preferences#privacy and changing the level of tracking protection to "Strict" causes Firefox to block all trackers, first party and third.

I suspect clicking on the shield icon at this point should reveal a UI that shows all the things that were blocked. This is not currently true; I'll file a bug if warranted.

All of this is to say, the UI may be a bit fluid right now, there may be some bugs, but the out-of-the-box, the browser is behaving exactly as we expect. It loads GTM; allows analytics; and does not raise big alarms about doing so. With strict mode on, it blocks GTM.

hoosteeno commented 5 years ago

Update: The reason we don't see any trackers blocked in strict mode is that Fx70 sends DNT headers when strict mode is enabled, and dubmo respects them WRT GA tags. Nothing was blocked because nothing was sent.

pmac commented 5 years ago

Doh! You're right. I was seeing googletagmanager.com being blocked because I have a local DNS tracking blocker. It works as you describe when I turn that off. My apologies for the churn.

alexgibson commented 5 years ago

See also: https://bugzilla.mozilla.org/show_bug.cgi?id=1576673

For posterity, it's bugs like this (^) that drive forward the affirmation to me that we should be investigating other solutions for our analytics. The page mentioned in that bug was a valulable source of measurement for website performance, since it was seen by so many people. Firefox traffic provides a large and diverse audience both in terms of geography and connectivity, and being able to measure and optimize for those users on our website remains an important feature.

Whilst this may not impact our measurement for Skyline, it is something we need to keep prioritized and on the roadmap for bedrock imho.

alexgibson commented 5 years ago

See also: https://bugzilla.mozilla.org/show_bug.cgi?id=1578872

alexgibson commented 4 years ago

Depends on https://github.com/mozilla/bedrock/issues/8482

alexgibson commented 4 years ago

Further reading: https://blog.convert.com/analytics-ab-testing-cookies-after-consent-in-europe.html

alexgibson commented 4 years ago

I'd like to keep this issue in our minds, so I'm sharing a couple of recent new products that aim to provide privacy-preserving analytics as an alternative to GA:

pmac commented 4 years ago

I also found https://usefathom.com/ via https://switching.software/replace/google-analytics/

alexgibson commented 4 years ago

Depends on https://github.com/mozilla/bedrock/issues/9028

alexgibson commented 4 years ago

Relevant: there are reports that Safari on the next version of macOS (Big Sur) may block GA by default (even with anonymised IP):

https://appleinsider.com/articles/20/06/22/safari-now-blocks-google-analytics-on-sites-new-privacy-report-feature-shows

alexgibson commented 3 years ago

This issue has stagnated somewhat since it was opened, because we couldn't relly get traction on it. But I'd like to reference a growing list of GA related bugs we now have on file.

Even without the GA blocking discussion, these bugs mean we are almost certainly under-reporting on our data (by how much, who knows!?). I'd love to put something like "Implement a 1st party analytics tool" on our roadmap, but it's something we'd need to plan / resource for.

/cc @slightlyoffbeat @pmac

slightlyoffbeat commented 3 years ago

Thanks Alex.

Since we are re-igniting this conversation:

Fully understanding our issues with GA is a first step towards either fixing or replacing our analytics solution.

maureenlholland commented 1 year ago

I believe this was completed in Asana task (Mozilla only): https://app.asana.com/0/1203970207351421/1203970207635769/f