matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.71k stars 2.62k forks source link

Referrer detection does not recognize subdomains when wildcards used in site URLs #17299

Open utrautmann opened 3 years ago

utrautmann commented 3 years ago

In some individual Matomo websites that I maintain the transitions report shows the own website as traffic source in category "Website". I mean with "own website" the measurable URL that was set up that the visitors use to access this website.

However, the number of accesses "From Website" is significantly lower than in the incoming traffic for internal pages (see screenshot please), so I assume that only very specific pageviews with my domain are counted under the category "From Websites" .

As URL was set up http://mydomain.com and (!) https://www.mydomain.com because as a workaround of Page Overlay bug #12418

grafik

The acquisition report does not show the "own website", correctly.

The expectation is that only external websites will be listed in the transitions report.

Is here a relationship to #15039 ?

diosmosis commented 3 years ago

Hi @utrautmann, sorry you're experiencing this. If your website URL is being listed as an external website here, then that must mean for one or more visits, that website URL is treated as the referrer URL. Would you be able to check if this is the case in the Acquisition > Websites report? Would you also be able to try and find a visit that has this as a referrer in the Visits Log (maybe by using the referrerUrl segment)?

I can think of one case where this might happen:

Finding a visit in the visits log would help determine if this is the case.

utrautmann commented 3 years ago

Hi @diosmosis , I tried to analyze it more closely.

In addition to the www domain, I also track a subdomain on the Matomo website. The configured URL is http://domain.de, so that I also include the subdomain in addition to www (sub.domain.de). I also track one other domain so that enableCrossDomainLinking is switched on.

_paq.push(["setDomains", ["*.domain.de", "*.seconddomain.de"]]);
_paq.push(["enableCrossDomainLinking"]);

Here are the results of my research that I have done.

  1. My www-domain is never being listed in the Acquisition > Websites report.
  2. But my subdomain is being listed in the Acquisition > Websites report.
  3. And my www-domain and my subdomain are being showed in the Visitor Log and in the Transitions report

I exported the visits in question to Excel. If the visitor now jumps back and forth between pages on www and the subdomain, different referrer entries are created in the visitor profile.

I found at least 4 different combinations of datas. There is even another external URL included (that has nothing to do with my Matomo or my mentioned seconddomain.de - see Profile 3) .

grafik

In Profile 1 the transition report looks so: grafik

I am sorry, unfortunately I have no precise identification of what happens when and at what point in time.

diosmosis commented 3 years ago

Hi @utrautmann, apologies for another late reply. From looking at the code it seems like the referrer is being set incorrectly when the Referer header is set to sub.domain.de. It should be recognized as part of the same site (if it's listed in the URLs portion of the site's settings). Can you tell me if your website is configured like this in Matomo:

image

? Or is sub.domain.de explicitly set?

I can definitely confirm that *.subdomain.de is not handled correctly in the tracker.

utrautmann commented 3 years ago

Hi @diosmosis , my website is configured im Matomo like this

The explicit www entry is configured because as a workarround for #12418 The tracking snippet contains following script:

<!-- Matomo -->
<script type="text/javascript">
  var _paq = window._paq = window._paq || [];
  _paq.push(["setCookieDomain", "*.mydomain.com"]);
  _paq.push(["setDomains", ["*.mydomain.com","*.seconddomain.com"]]);
  _paq.push(["enableCrossDomainLinking"]);
  _paq.push(['trackPageView']);
  _paq.push(['enableLinkTracking']);
...
<!-- End Matomo Code -->
diosmosis commented 3 years ago

@utrautmann Are you able to add the sub.domain.com URL that appears as a referrer? I believe that might workaround the problem.

utrautmann commented 3 years ago

@diosmosis : Where should I exactly add the subdomain-URL? In the backend configuration or/and in the tracking script ("setDomains..."-method)? Thank you.

diosmosis commented 3 years ago

@utrautmann in the backend configuration. This is what's used when detecting whether a referrer URL is from a the same site or somewhere else.