Investigate alternatives to Sourcebuster (the JS library we're currently using)

[rawdreeg]

Description

The extension currently relies on Sourcebuster JS to track the sources of sites' visitors and store the data in the cookies for further analysis.

Given the implications of using cookies in this process, it seems that we’ll want to avoid them.

More info: pe2C5g-ur-p2

Tasks.

• [x] Look into Trafficsourcetracker.js
• [x] Look into trak.js
• [x] Look into visitdatajs
• [x] Document findings

[MatthiasReinholz]

@rawdreeg given the simplicity of cookies over any alternative solution, I'm not convinced anymore that a first version needs to avoid cookies. Also, the market seems to kill cookies much slower than expected. As of today, they're still the industry standard.

If we want to avoid them, the alternatives would require a server-side process.

Not sure if you have already found a library that would handle this. This is how I've done this in the past:

• enqueue a JS to the frontend on all pages that more or less does nothing else than calling an endpoint upon every single page load
• in the endpoint, access the superglobals to retrieve IP, user agent, full request URL, referrer
• add some processing of the user agent (e.g. via https://packagist.org/packages/matomo/device-detector)
• add some processing of URL params for the request URL (especially about the UTM params)
• create something like a session ID based on the IP, the user agent, and the date
• store the dataset somewhere (e.g. in a database table. We've recently also discussed the option to use Transients for this)

The downsides of this specific server-side approach are: a) the additional load to the server b) the possibility to identify two different visitors as the same (if both are using the exact same device type with the exact same browser version behind the same IP).

I think with the goal to keep things simple, we may still start with a cookie-based solution first before entering the more adventurous area of server-side tracking.

[rawdreeg]

Thanks, @MatthiasReinholz. You hit on many of the points I planned on covering in my P2 post to wrap up the task.

If we want to avoid them, the alternatives would require a server-side process.

The closest I could find to achieving this is visitdata -- which is a very simple GA emulator that should provide a way to get the source and medium data programmatically through JavaScript. E.g.:

// Example raw object returned by visitdatajs
{
    "this_hostname": "https://wpsandbox.ngrok.io",
    "this_domain": "ngrok.io",
    "referring_hostname": "https://wpsandbox.ngrok.io/",
    "referring_domain": "ngrok.io",
    "query_string": "",
    "utm_tags": null,
    "url_params": null,
    "paid_url_data": null,
    "organic_search_data": null,
    "referral_data": {
        "medium": "referral",
        "source": "ngrok.io"
    }
}

There's the question of session data, for which I agree with you that the best option would be to get these values server side. I believe these data could be retrieved and cached as transient options. We do this

A third approach would be to implement, along with the server-side approach, a rudimentary browser fingerprinting mechanism to help tell users apart.

I think with the goal to keep things simple, we may still start with a cookie-based solution first before entering the more adventurous area of server-side tracking.

Yes, source buster may very well be perfect for our current needs. I will document my findings so far and open them up for discussion.

Again, Thanks,

R

[rawdreeg]

I posted a full investigation of potential alternatives to sourcebuster on pe2C5g-Cv-p2.

I will mark this task as complete.