search

matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.89k stars 2.65k forks source link

Piwik block by AdBlock even after renmaing piwik.php piwik.js #7364

Open sambuev opened 9 years ago

sambuev commented 9 years ago

Hello, I have Piwik installed on my hosting with secret address: interes.site.com/sm/ and I have renamed piwik.php piwik.js to sm.php sm.js and I though it cannot by caught by AdBlock but unfortunately AdBlock on my Safari browser easily blocking Piwik and many other users with their Adblock (on Chrome, Firefox) out of my view :(

screen shot 2015-03-04 at 18 21 38

What can I do to avoid blocking Piwik? Is it possible to avoid it?

MESWEB commented 9 years ago

I got this same error but I think this is difficulty for dev piwik

kylekatarnls commented 9 years ago

I use rewriting and replace /piwik.php with juste /p. Works like a charm.

MESWEB commented 9 years ago

Can You give any solution? I trying this but without success with uBlock - https://www.codelibrary.me/tag/ad-block/

kylekatarnls commented 9 years ago

This is what I did (but with a proxy-pass in the same domain).

First remove "analytics" in the URL when you enter the rewrite rule and any word that can be seen as ads.

Then if it's not enough, use a proxypass (http://httpd.apache.org/docs/2.2/fr/mod/mod_proxy.html) on your own domain, so your URL to call Piwik will just look like "/p" and "/j"

(function() {
    _paq.push(['setTrackerUrl', '/p']);
    _paq.push(['setSiteId', 1]);
    var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
    g.type='text/javascript'; g.async=true; g.defer=true; g.src='/j'; s.parentNode.insertBefore(g,s);
  })();
Joey3000 commented 9 years ago

Guys, have you tried just replacing "piwik.php" and "piwik.js" in the tracking code with "js/", as described in https://github.com/piwik/piwik/blob/master/js/README.md? No .htaccess rewriting, file renaming, etc. needed.

See also: http://forum.piwik.org/read.php?2,83940,125929 http://forum.piwik.org/read.php?4,14971

I've been using that for years. Just make sure that the "piwik" string (or any other string blocked by the ABP) doesn't appear anywhere in the URL (i.e. domain name, directories). After coming across this issue report, I've just tried it again with ABP on Firefox and it doesn't block it. I checked the ABP "easyprivacy" list again, and it only checks for the "piwik" string anywhere in the URL.

P.S.: Another solution is in https://piwik.org/faq/how-to/#faq_132, but that would require the server being able to make outbound connections.

MESWEB commented 9 years ago

Non of all solution what You give not working with uBlock! I think the uBlock is hard to defeat. Only solution is use any server-side tracker. Piwik should use server-side tracking by default.

Joey3000 commented 9 years ago

@MESWEB From https://addons.mozilla.org/en-us/firefox/addon/ublock-origin/:

Out of the box, these lists of filters are loaded and enforced:

- EasyList
- Peter Lowe’s Ad server list
- EasyPrivacy
- Malware domains

Any chance your Piwik URL doesn't pass any of those?

Joey3000 commented 9 years ago

There is following investigation necessary:

And if the block can be URL parameter based, it could be the time for a Piwik plugin where the user could freely change them. (I don't know how deep into the rabbit hole one would need to go to do that.)

kylekatarnls commented 9 years ago

@MESWEB Piwik was using server-side (it was called phpMyVisites), but for many reasons (performances, security) it has been switched to client-side and it's a good thing. I use proxypass + url-rewriting (it's exactly equivalent to the solution mentionned by @Joey3000, but it is faster because it does not use PHP) and uBlock does not block this solution.

MESWEB commented 9 years ago

Yes I know it and I try changing url or filenames and dynamic changed filenames with .htaccess with RewriteRuele but without effect. I saw something strange - Any of my browser (Opera, Chrome, Waterfox) hasn't be counted by piwik but when Author of the PIWIK goes on my site I see his log in PIWIK. So my question is WTF? I'm not only one with this strange bug.

kylekatarnls commented 9 years ago

Maybe you missclicked on uBlock and add a filter in your browser try to reset uBlock. Else, if all your stuff is on the same domain, it should not be blocked as uBlock does not search in the response body but only in request URLs.

bdore commented 7 years ago

Ok so I was able to go around Opera's built-in adblocker with some major changes to PiWik. I'm still testing to see if it holds together after deploy. This was tested only on a local server and PiWik self-hosted:

Changes I made to the PiWik installation files:

Do all this at your own risk, this is totally experimental. This probably breaks something in PiWik. Not thoroughly tested.

novakin commented 7 years ago

@bdore would be great to be able to do that trough a plugin

sboesch commented 7 years ago

Keep in mind, instead of renaming any files, you should also be able to create symlinks:

$ ln -s piwik.js p.js && ln -s piwik.php p.php

This is much cleaner and at least uBlock seems to allow these requests as far as I checked for now.

FabriceSalvaire commented 7 years ago

A lot of things are wrong here !

EasyList contains these basic rules :

Thus we have just to change in the tracking script

And a two URL rules in the webserver to map them to the original paths.

It will work until the parameter list doesn't contain evident keywords like stat, log, track ... Blockers can try to be clever, but they will never broke the www.

arnowelzel commented 7 years ago

If you use the "proxy" script to hide the real URL of your Piwik server (see https://github.com/piwik/tracker-proxy) you can also use that to hide parameter names as well.

When sending the JavaScript, just do a str_replace for "action_name=" to something else, e.g. "aname=" and then rename the $_GET parameter back to the original name. You should also rename this proxy script from "piwik.php" to something else.

So - first replace "action_name" in the provided JavaScript from the Piwik server:

        if ($piwikJs = $content) {
            echo str_replace('"action_name="', '"aname="', $piwikJs); 
        } else {

And this to accept "aname" as parameter but forward it as "action_name" in the server side CURL request to the Piwik server:

if(isset($_GET['aname']) {
        $_GET['action_name'] = $_GET['aname'];
        unset($_GET['aname']);
}
foreach ($_GET as $key => $value) {
    $url .= urlencode($key ). '=' . urlencode($value) . '&';
}

But be aware, that even the new parameter name "aname" may once get into filter lists as well - so you may have to use something else.

omarr1000 commented 7 years ago

@arnoweizel: where do place this code? In which file? I tried in piwik.php (from tracker-proxy), but no success, nothing is replaced, still have "action_name" thx for an advice in advance

arnowelzel commented 7 years ago

@omarr1000: then you did something wrong. See the script here, which I use on my own server (https://arnowelzel.de):

https://arnowelzel.de/samples/piwik-tracker-proxy.txt

omarr1000 commented 7 years ago

thx! now I got it, did write the code at the wrong line )-;

londonuk371 commented 7 years ago

I had to do 2 things: -First the symlinks $ ln -s piwik.js p.js && ln -s piwik.php p.php like proposed by @sboesch

ublocOrigin was blocking ?action_name=

hparadiz commented 6 years ago

Matomo tracking information should be obfuscated both to avoid being picked up by ad blockers but also for the security of the person being tracked! The current tracking URL is obnoxious with it's GET variables being sent over HTTP in plaintext.

Here's an example: https://domain/a.php?action_name=site&idsite=1&rec=1&r=058405&h=11&m=1&s=31&url=https%3A%2F%2Fdomain%2F&_id=9b550a96cfa8d490&_idts=1523890834&_idvc=1&_idn=0&_refts=0&_viewts=1523890834&send_image=1&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=0&java=0&gears=0&ag=0&cookie=1&res=1920x1200&gt_ms=27&pv_id=Zf3r32

Matomo should generate a short random string and automatically create this .htaccess file if mod_rewrite is detected:

RewriteEngine On

RewriteRule ^a.js piwik.js [L,QSA]
RewriteRule ^a.php piwik.php [L,QSA]

and then when it provides you the code snippet should automatically have this url end point used.

Proposed Solution $_GET variables can be json encoded and then encrypted with a key generated for each matomo install. You provide the key to the js tracking file. JavaScript then instead provides the data like so: a.php?VGhpcyBpcyBhbiBlbmNvZGVkIHN0cmluZw==

On the PHP side you would simply base64 decode then json_decode then just array_merge with $_GET.

arnowelzel commented 6 years ago

The key needs to be embedded in the JavaScript - otherwise it is not possible to encrypt anything on the client. So you don't gain any privacy, as anyone else could also send a request to the proxy script and find out, what key is used.

select commented 6 years ago

I managed to unblock uBlock by creating the symlinks for `piwik.(js|php) and changing pwik.js

"action_name=

to 

"a=b&action_name=

no need to rename variables @arnowelzel @bdore @londonuk371

select commented 6 years ago

What also might work is creating a symlink like 

mkdir foo
ln -s piwik.php foo/index.php

then point the tracking script to /foo, no need to mess with routes

isopetalous commented 6 years ago

Do not use "piwik" as the folder name, rename it to something else. No need to rename the files or make rewrite rules. Just modify the tracking code and replace all entries of "piwik.php" and "piwik.js" with "js/". js/index.php will serve both php and js files for you.

BurninLeo commented 6 years ago

If you use the "proxy" script to hide the real URL of your Piwik server (see https://github.com/piwik/tracker-proxy) you can also use that to hide parameter names as well.

That's just a cool idea, thanks! It's even possible without creating a new URL, if you just use require('piwik.php') below setting the $_GET variable.

For those who are still searching, here are two PHP scripts to put in the piwik directory:

pi-js.php

<?php
$jsFile = file_get_contents('piwik.js');
echo strtr($jsFile, [
  '"action_name="' => '"myAC="',
  'piwik.php' => 'pi-php.php'  // Not sure, if this is really necessary
]);
?>

pi-php.php

<?php
if (isset($_GET['myAC'])) {
        $_GET['action_name'] = $_GET['myAC'];
        unset($_GET['myAC']);
}
require('piwik.php');
?>

Then just replace piwik.php in the Matomo code by pi-php.php and piwik.js by pi-js.php. No symlinks required, but you can use some to "hide" the name of the directory. This solution should even survive the next update of Matomo.

johackim commented 5 years ago

This works for me :)

sed -i -e 's/action_name=/a=b\&action_name=/g' matomo.js
ln -s matomo.js m.js && ln -s matomo.php m.php
<script type="text/javascript">
  var _paq = window._paq || [];
  _paq.push(['trackPageView']);
  _paq.push(['enableLinkTracking']);
  (function() {
    var u="//stats.domain.com/";
    _paq.push(['setTrackerUrl', u+'m.php']);
    _paq.push(['setSiteId', '1']);
    var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
    g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'m.js'; s.parentNode.insertBefore(g,s);
  })();
</script>

Thanks @select & @sboesch

w-biggs commented 5 years ago

I found that the easiest thing to do overall is make two tiny changes:

  1. Run sed -i -e 's/action_name=/a=b\&action_name=/g' matomo.js in the directory Matomo is installed in.
  2. Replace the g.src=u+'m.js' in the tracking code with g.src=u+'js/'.

That's it - now Matomo works, even if uBlock is installed.

timbowhite commented 5 years ago

@ston3o @w-biggs

  1. chmod 444 matomo.js or chattr +i matomo.js

The Auto-Archiving cron runs periodically (either from your browser or via crontab depending on your config) and regenerates matomo.js, wiping out any changes you've made to it. The above should lock it down.

mattab commented 5 years ago

regenerates matomo.js, wiping out any changes you've made to it. The above should lock it down.

To have your changes included in the regenerated matomo.js file, you can put the changes in the file matomo/js/piwik.min.js which is the "Source" file read to regenerate the minified file.

inputsh commented 5 years ago

There's an additional rule in EasyPrivacy:

&idsite=*&send_image=$image

This makes it slightly trickier because doing a similar sed operation to the one mentioned by @ston3o and @w-biggs will not work thanks to that *.

I guess the alternative approach would be to either rename one of those two variables or to switch the order of send_image and idsite.

45236 commented 4 years ago

There's an additional rule in EasyPrivacy:

&idsite=*&send_image=$image

This makes it slightly trickier because doing a similar sed operation to the one mentioned by @ston3o and @w-biggs will not work thanks to that *.

I guess the alternative approach would be to either rename one of those two variables or to switch the order of send_image and idsite.

that is exactly my problem. How do I rename that or how do I change the order?

sgiehl commented 4 years ago

@45236 you can use POST requests for tracking. That will make it impossible to replay tracking logs if something goes wrong with it, but should circumvent the rule. See https://developer.matomo.org/api-reference/tracking-javascript#advanced-uses

arnowelzel commented 4 years ago

JFTR: the modified proxy script still works. If you need to replace additional parameters, like "idsite" etc. - just go ahead and do it: https://arnowelzel.de/samples/piwik-tracker-proxy.txt

Live in action: https://arnowelzel.de

ikmolbo commented 4 years ago

There's an additional rule in EasyPrivacy:

&idsite=*&send_image=$image

This makes it slightly trickier because doing a similar sed operation to the one mentioned by @ston3o and @w-biggs will not work thanks to that *. I guess the alternative approach would be to either rename one of those two variables or to switch the order of send_image and idsite.

that is exactly my problem. How do I rename that or how do I change the order?

I managed to do this by chaining two search and replace commands as follows:

sed -i -e 's/&send_image=0/\&/g' js/piwik.min.js; sed -i -e 's/action_name=/send_image=0\&action_name=/g' js/piwik.min.js

This will reorder send_image=0 and also overcomes the filter on action_name. You will need to call ./console custom-matomo-js:update once you have done this to ensure these changes are picked up.

Having done this, I am now seeing visits recorded even when using ad blockers.

45236 commented 4 years ago

@ikmolbo nice and in which file do I have to replace that?

jb-ks commented 4 years ago

@arnowelzel The proxy with your changes works like a charm, but I wonder how to get the opt-out working.

I know, that in your proxy-file (based on the old matomo-proxy) is no support für the opt-out frame.

So i tried to display the opt-out frame exactly as before using the proxy (accessing index.php in the matomo installation). The frame is displayed, you can opt-out and later opt-in again, but - even if it seems to be opted out - the tracking is running.

Note:

arnowelzel commented 4 years ago

@jb-ks Opt-out can't work because the ignore-cookie is set for the Matomo URL and not for the website which hosts the proxy script. The proxy script has no chance at all to check wether there is a ignore-cookie set or not, because the idea of this script is not to call the Matomo URL ever using the browser to avoid getting blocked.

In my case I just don't offer opt-out since I use Matomo without any cookies at all on my website:

<script>
    var pkBaseURL = "https://arnowelzel.de/";
    document.write(unescape("%3Cscript src='" + pkBaseURL + "pwproxy.php' type='text/javascript'%3E%3C/script%3E"));
</script><script>
    try {
        var piwikTracker = Piwik.getTracker(pkBaseURL + "pwproxy.php", 1);
        piwikTracker.trackPageView();
        piwikTracker.enableLinkTracking();
        piwikTracker.disableCookies();
    } catch( err ) {}
</script><noscript><p><img src="https://arnowelzel.de/pwproxy.php?ids=1&amp;rec=1" style="border:0" alt="" /></p></noscript>

Without any tracking cookies Matomo won't recognize returning visitors - but this is not important for me anyway. It's enough to see how many visits and page views I got and from which external URL they come (which only requires the HTTP referrer but no tracking cookies) and link tracking inside the website still works as this only requires the IP address during a session (all requests from the same IP address within a certain time period are considered to be the same visitor).

And yes - of course IP addresses are anonymized in Matomo on my server (the last two bytes are set to 0). But even anonymized IP addresses are sufficient for link tracking without tracking cookies.

And from a privacy standpoint: yes, I thinks this is legitimate, since accessing a website requires an active connection from the visitor anyway. A visitor can't say "I want to see your website but I don't want the webserver to know about that".

jb-ks commented 4 years ago

@arnowelzel Thanks for your explanation.

I have therefore decided to include a local alternative. The solution by Clive Beckett at https://github.com/clivebeckett/matomo-opt-out works very well after some initial detail problems.

suppadeliux commented 4 years ago

Hello, I tried some of the solutions from this issue, and somehow now my ad blocker is not blocking the request anymore!!

I created links with the command ln -s piwik.js p.js && ln -s piwik.php p.php & ln -s matomo.js m.js && ln -s matomo.php m.php, and also changed the action_name and idsite for something else.

And since I am using the angular plugin for matomo, I am overloading some of the code so I can change the _paq.push(['setTrackerUrl', u + 'p.php']); and the g.src = !!scriptUrl ? scriptUrl : u + 'p.js';

BUT, now I have this error: https://mysite.com/p.php?my_action.......... 403 (Forbidden)

I guess that the tracker cannot find the file p.php. But i have no idea where to change this.

For example if I type in my browser the matomo instance mysite.com/piwik.php or mysite.com/matomo.php . I have the message (This resource is part of Matomo. Keep full control of your data with the leading free and open source web analytics & conversion optimisation platform.)

But if I type mysite.com/p.php, this doesnt work and sends an error.

I am so confused that I dont really know what to do, maybe the solution is pretty easy but I cant see that.

Thank you for your help.

UPDATE - 4th august: Since my matomo instance was served by an nginx web server, I updated the configuration file, and added the name file m.php in sites-available and that's it!.

Maybe this will be helpful for someone.

Tealk commented 3 years ago

I really do not know how to get around this grafik

arnowelzel commented 3 years ago

I really do not know how to get around this grafik

By using a proxy script which takes parameters with other names and forwards this to the real script of Matomo as suggested already in 2017.

Tealk commented 3 years ago

By using a proxy script which takes parameters with other names and forwards this to the real script of Matomo as suggested already in 2017.

Does this still work, this is already a good 4 years old

arnowelzel commented 3 years ago

By using a proxy script which takes parameters with other names and forwards this to the real script of Matomo as suggested already in 2017.

Does this still work, this is already a good 4 years old

Yes - it still works, even with a current version of Matomo. The script I posted here is still in use on my own website and many others:

https://arnowelzel.de/samples/piwik-tracker-proxy.txt

kylekatarnls commented 3 years ago

It was even mentioned yet in 2015 ^^ https://github.com/matomo-org/matomo/issues/7364#issuecomment-140331043

Principle is that there are plenty different ad blockers trying to use different detection ways, you have to let nothing making possible to link the script to Matomo.

Tealk commented 3 years ago

https://arnowelzel.de/samples/piwik-tracker-proxy.txt

but the tracking code seems to have changed and i am not really fit in javascript

<!-- Matomo -->
<script type="text/javascript">
    var _paq = window._paq = window._paq || [];
    _paq.push(['requireCookieConsent']);
    _paq.push(['setSecureCookie', true]);
    _paq.push(['trackAllContentImpressions']);
    (function() {
        var u = "https://trackedsite.com/";
        _paq.push(['setTrackerUrl', u + 'indexx.php']);
        _paq.push(['setSiteId', '6']);
        var d = document,
            g = d.createElement('script'),
            s = d.getElementsByTagName('script')[0];
        g.type = 'text/javascript';
        g.async = true;
        g.src = u + 'indexx.js';
        s.parentNode.insertBefore(g, s);
    })();
</script>
<noscript>
    <p><img src="https://trackedsite.com/indexx.php?idsite=6&amp;rec=1" style="border:0;" alt="" /></p>
</noscript>
<!-- End Matomo Code -->

how do I have to adjust this now?

arnowelzel commented 3 years ago

https://arnowelzel.de/samples/piwik-tracker-proxy.txt

but the tracking code seems to have changed and i am not really fit in javascript

The proxy script does not need to be changed - it just loads the content of piwik.js from Matomo and replaces a number of parameter names so they can be used with a different name.

On my site, the script is then used this way:

<script>
    var pkBaseURL = "https://arnowelzel.de/";
    document.write(unescape("%3Cscript src='" + pkBaseURL + "pwproxy.php' type='text/javascript'%3E%3C/script%3E"));
</script><script>
    try {
        var piwikTracker = Piwik.getTracker(pkBaseURL + "pwproxy.php", 1);
        piwikTracker.trackPageView();
        piwikTracker.enableLinkTracking();
        piwikTracker.disableCookies();
    } catch( err ) {}
</script><noscript><p><img src="https://arnowelzel.de/pwproxy.php?ids=1&amp;rec=1" style="border:0" alt="" /></p></noscript>

And this works with Matomo just fine. No need for any JavaScript changes. Just put pwproxy.php with the content as shown in https://arnowelzel.de/samples/piwik-tracker-proxy.txt and use it as shown above.

I also have page which shows the statistics of my site by using the Matomo API: https://arnowelzel.de/en/tools/statistics

Tealk commented 3 years ago

is it possible to enable _paq.push(['setSecureCookie', true]);?

and i have some functions like:

function showHome() {
  ...
  _paq.push(['setDocumentTitle', 'Home']);
  _paq.push(['trackPageView']);
}

Also I get the message and I do not want to turn off nosniff The resource of "https://trackedsite.com/mproxy.php" was blocked because of a MIME type conflict ("text/html") (X-Content-Type-Options: nosniff).

arnowelzel commented 3 years ago

You can substitude the _paq calls with the respective Methods of piwikTracker, e.g.

piwikTracker.setDocumentTitle('Home');
piwikTracker.trackPageView();

Note: piwikTracker.trackPageView(); is already included in my code.

See the available methods of the "old" API at https://developer.matomo.org/api-reference/tracking-javascript.

About the cookie consent: I don't set a cookie at all (piwikTracker.disableCookies();), so I also don't need any consent for this. In my experience this is good enough to get useful statistics for a website. I don't need information about returning visitors. However - if you use cookies you should also implement some kind of cookie consent dialog on your website to fulfil GDPR requirements.

Tealk commented 3 years ago

However - if you use cookies you should also implement some kind of cookie consent dialog on your website to fulfil GDPR requirements.

I have written a very nice one for myself.