LAtimes.com - filter rules to fix new native ads

[lukemulks]

3 different flavors of new, native ads discovered as of this morning on latimes.com - being tracked and investigated here: https://github.com/brave/browser-laptop/issues/7454

These are different and new, unrelated to recent PR that was merged last week.

2 different versions of "sponsored content" ads discovered this morning on Windows, which the rules below have confirmed resolving when tested in about:adblock.

||aggrego.org^$script,image, domain=latimes.com
||adserve.postrelease.com^$script,image, domain=latimes.com
||troncdata.com^$script,image, domain=latimes.com
||polarmobile.com^$script,image, domain=latimes.com
||ntv.io^$script,image, domain=latimes.com
||gigya.com^$script,image, domain=latimes.com

A 3rd issue has been reported from MacOS, but has not yet been reproduced in Windows. Still investigating that.

Will submit a PR for adding the rules above to solve for 2/3, and will follow up here if the third issue is also resolved once I'm able to verify from MacOS.

[lukemulks]

PR submitted: https://github.com/brave/adblock-lists/pull/9

[lukemulks]

Confirming that after testing on iOS, no longer seeing the ads display that the rules above resolve.

The third item I was unable to repro was confirmed as a shield-down observation, so considering the PR above the fix for the issues reported today.

[lukemulks]

@bbondy confirming that all the rules merged above are functioning as expected in 0.13.5, except for one which I made an error on.

I've corrected the error, and submitted this PR to resolve. https://github.com/brave/adblock-lists/pull/10

Apologies for the oversight. Tested the update with the corrected version in about:adblock, and it fully resolves the issue.

[matt2000]

Hi, I'm the developer for troncdata.com (an internal service of LATimes / Tronc) and I want to let you know that it does not serve advertising at all, native or otherwise. Can you please remove it from your block list? Thanks.

[lukemulks]

We're going to have to agree to disagree in this instance.

I am going to strongly object to removing troncdata from the blocklist.

I've reversed the implementation, and troncdata appears to be used from the 1st party level with an ad widget, invoking an infusion script function with nested timing sequences for matching page data and user IDs to delayed ad calls that bind and replace to an html <aside> container in the right rail. data-state and other data-* attributes are used in a clever way in this implementation.

Gigya appears to be the user-match/attribution vendor involved in this one. Forbes and others use Gigya, which follows a similar pattern. It appears other vendors are involved as well (teads, etc.) at the advertiser level, but it appears that Gigya directly correlates with the native widget in the right rail (dispatch references for the widget in the script)

Request 1: http://recommend.troncdata.com/js/widget.js

Response:

trb.runInfuse.push(function() {
var tries = 0;

var related = $('.trb_ar_rail div[data-eg-type=related]');

var isShown = false;
function showRecommendations() {
  related.css('opacity', 1);
  isShown = true;
}

function waitForData() {
  if(trb.hasOwnProperty('data')
     && trb.data.hasOwnProperty('page')
     && trb.data.page.hasOwnProperty('slug')
    ){
    (function($) {

      if (!localStorage.getItem('recsys_override') && location.hash.includes('enable-troncrecs')) {
        localStorage.setItem('recsys_override', 99);
      }

      var userId = localStorage.getItem('recsys_override') || trb.cookie.get('uuid') || localStorage.getItem('recsys_id');
      if (!userId) {
        var userId = Math.random() + Date.now();
        localStorage.setItem('recsys_id', userId);
      }
      var params = [userId, trb.data.page.slug];
      $.ajax('https://recommend.troncdata.com/recommendations/'
              + params.join('/'), {
        'dataType': 'json',
        'crossDomain': true,
        'complete': showRecommendations,
        'success': function(data) { if (data['markup'] && !isShown) related.replaceWith(data['markup']);}
      });
      setTimeout(showRecommendations, 1500);
    })(i$.jQuery);
  } else if (tries < 10) {
    tries = tries + 1;
    setTimeout(waitForData,50);
  } else {
    showRecommendations()
  }
}
waitForData();
});

XHR Request: https://recommend.troncdata.com/recommendations/57669011-a2b8-42a1-968a-23695a7422ee/la-me-ln-winter-temperature-records-20170310

Response: {"recsys": "control"}

Gigya script: http://cdn.gigya.com/js/gigya.js

Walkthrough to illustrate the crawl (sans-blocking)

Tribune infusion runs...

troncdata passes values from the page, but also sets the slug data as the ad unit IDs for the ad widget toward the bottom of the screencap.

Continue the crawl, and discover the same troncdata widget script with gigya passed through as the mediaConductor, which also contains a site ID assigned for LA Times from the vendor (gigya) within the same <aside> html tag for the right rail (matching <aside id=> value.

At a minimal level_, this is working in conjunction with the ad product integration on the page (and likely, on other tronc sites). UUIDs are passed in the string (57669011-a2b8-42a1-968a-23695a7422ee) which data management platforms can match users to. With this blocked as-is, no webcompat issues have been reported in over a week, and the content that does load in the right rail slot doesn't appear to show any issues.

We want to block this imo, cc: @bbondy

[bbondy]

Thanks @lukemulks

[matt2000]

@lukemulks Your analysis is incorrect, and I'd like to help clarify the purpose of this widget. I wrote widget.js and the backend service at recommend.troncdata.com that it communicates with. I'm happy to be 100% transparent about what's happening here. (As you can see, the script is not even minimized, let alone obfuscated.) I work directly for Tronc / LA Times, and have since July 2016. (I.e., I'm not a contractor or third-party service provider. I collaborate with the newsrooms on this project, not ad sales.)

• There is no connection to gigya at all, and it's not clear to me how you got the idea that there was. The <script> blocks in your 3rd screen shot are all completely independent. I don't even know what "mediaconductor" is (many parts of the page are owned by different departments), but you can clearly see it's not used by my script.

• I directly and personally control troncdata.com, not gigya or any ad provider. I can think of various ways to prove that to you, if needed. (E.g., adding DNS records or custom HTTP responses, although I don't have convenient access to modify the domain WHOIS, but could arrange it if that would satisfy you.)

• The ID used is generated by Ensighten, and is merely a convenient identifier for matching users across our analytics platforms.

• We use the User UUID only to track what user's have read so as to not show them content they've already seen. In the near future, we expect to use it to recommend to them content that is similar to content they've read, or of interest to other users who've read similar content. (Collaborative filtering, etc.)

• This widget does not share data with any third-party. In fact, it doesn't even persist it's own data beyond the lifetime of an individual server thread. (Long-term user behavior is gathered from regular web analytics, so we only care about real-time activity, and update form the analytics data when it becomes available in batch.)

• We're currently A/B testing against the original <aside> block. The fact you see the response {"recsys": "control"} just means you're in the control group (based on UUID, along wiht 70% of users as of today) and so the HTML is unmodified. As you read the JS, you know how to opt yourself into the "B" group if you want to see what the returned mark-up looks like for users in the test-group. (It's identical, except for the articles that are selected to appear.)

• We'll probably ultimately be doing this on the first-party domain, so there's really no point in cluttering your list. I don't expect you to believe me, but you shouldn't have to, because the purpose of this widget is not to serve advertising. It's only for content recommendation, i.e., legitimate news articles by our journalists. Please spend your time knocking down the Taboola's and Outbrain's of the world. We're just trying to help users find quality journalism content that interests them, from a real news agency.

Please let me know if there are any other questions I can answer to set your concerns at ease.