The tracking cookies are no longer blocked on Firefox 69 and 70, if you are enrolled in the "Experiment" branch

[SoftVision-CosminMuntean]

[Affected Versions]:

• Nightly 70.0a1
• Firefox Beta 69.0b12

[Affected Platforms]

• All Windows
• All Mac
• All Linux

[Prerequisites]

• Have a new Firefox profile created.
• The latest "etp-search-volume-study@shield.mozilla.org-1.2-signed" version is installed.
• You are enrolled in the "Experiment" branch.

[Steps to reproduce]:

1. Open Firefox Beta 68.0b14 with the profile from prerequisites.
2. Update the browser to Firefox 69.0b12.
3. Navigate to "about:url-classifier" page and update the lists.
4. Navigate to "https://senglehardt.com/test/cookie_restrictions/simple_iframe.html" page.
5. Observe the iframe.

[Expected results]:

• The value generated for "Your current non-HTTPOnly cookies are:" is not displayed.

[Actual results]:

• The value generated for "Your current non-HTTPOnly cookies are:" is displayed.

[Notes]:

• When trying to update the "mozilla" provider list from about:url-classifier" page, you get an update error (NS_ERROR_FILE_IS_LOCKED).
• Attached a screen recording with the issue:

[TanviHacks]

Looks like trackertest.org is not being treated as a Tracker, but rather just a Third Party, when we update from Firefox 68 to 69. @englehardt or @bakulf can either of you explain this?

[englehardt]

The trackertest.org domain isn't on the Disconnect lists. Instead, it's hard-coded in Firefox as part of the moztest-track-simple table (https://searchfox.org/mozilla-central/rev/30b01f4f60dbcbd6b01500a26b3100c28005cf62/toolkit/components/url-classifier/LookupCache.cpp#948-952). It looks like this table used to be named test-track-simple (no moz prefix), which is the name set by the extension. This table was renamed in Firefox 69. See: https://searchfox.org/mozilla-central/diff/d1ed4c245a97504ca30be076c3bbdbe0035da6c4/modules/libpref/init/all.js#5220, as part of https://phabricator.services.mozilla.com/D34576.

Thus, it makes sense the Firefox 69 no longer blocks the test domain. That's not a cause for concern though, as the rest of the lists should still be active and thus the study data should be valid.

It is concerning that you see an error on list update, because that means users aren't actually getting the lists. Maybe that's causes by trying to do an update with a non-existent list? If you want a test page to check for the normal lists, use this one instead: https://senglehardt.com/test/trackingprotection/test_pages/tracking_protection.html. This uses test domains that are injected during list generation.

Also note that not blocking trackertest.org may have implications as to whether users see the tracking protection UI tour. See: https://github.com/mozilla/bedrock/issues/6398#issuecomment-435797766.

[SoftVision-CosminMuntean]

I have used the https://senglehardt.com/test/trackingprotection/test_pages/tracking_protection.html test page and compared the results from Firefox 68.b014 with 69.0b12 when the ETP Search Volume study is installed and enrolled in the "Experiment" branch. It seems that there are no differences between these versions using the test page.

@TanviHacks @englehardt does this mean that this behavior will not affect the users in any way if they are enrolled in the "Experiment" branch?

Also, regarding the error on list update, it is only reproducible after updating the browser even if you have or not the experiment installed. The issue is not reproducible with new profiles on Firefox 69. Should we log this issue in other repository or Bugzilla?

[englehardt]

@TanviHacks @englehardt does this mean that this behavior will not affect the users in any way if they are enrolled in the "Experiment" branch?

I think it might impact whether users see the UI tour, since the UI tour seems to use the tracker.org domain Are we still enrolling users? If not, I suspect all users would have seen the tour by then. cc/ @ericawright since Tanvi is on PTO for a while.

Aside from that, the moztest-track-simple list only contains trackertest.org and itisatracker.org domains. As far as I know, these domains don't have any use outside of the UI tour page. As long as the test page gives the following output, you know Firefox is loading the ads-track-digest256,social-track-digest256,analytics-track-digest256,content-track-digest256 lists, which actually contain the tracking domains. You can also do an additional sanity check by visiting a site like cnn.com and ensuring it actually finds trackers.

Also, regarding the error on list update, it is only reproducible after updating the browser even if you have or not the experiment installed. The issue is not reproducible with new profiles on Firefox 69. Should we log this issue in other repository or Bugzilla?

Definitely! This seems pretty serious. Would you mind to file a bug in the Anti-tracking component on Bugzilla and cc me / needinfo baku? Thank you.