matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.9k stars 2.65k forks source link

Invalid idSite: '0' #12264

Closed SVyatoslavG closed 5 years ago

SVyatoslavG commented 7 years ago

Hi everyone. I am getting this error very ofter. Error in Piwik (tracker): Invalid idSite: '0', referer: http://www.example.com/my-page.html I have site with about 100.000 pageviews per week and this error happens ones every 10-15 second in the apache log. It definitely comes from core/Tracker/Request.php core/Tracker/Request.php Is there any solution for this? I believe its a bug because I have only 1 site with PIWIK tracking code installed and about half of traffic actually makes on the PIWIK end.

sgiehl commented 7 years ago

Maybe the tracking code is not integrated correctly at some point. Would you mind sharing the site you are tracking. I would have a short look if everything is correct...

Sudrien commented 6 years ago

Since the original poster didn't provide, here's my examples from this evening, with slight redaction:

2018/04/03 23:41:55 [error] 22749#0: *2338 FastCGI sent in stderr: "PHP message: Error in Matomo (tracker): Invalid idSite: '0'" while reading response header from upstream, client: 182.253.139.67, server: m.sudrien.net, request: "GET /js/?pg=*****&appid=5&lt=log&pv_rf=https%3A%2F%2Fwww.google.co.id%2F&uuid=e880d415-6fcd-4963-9953-ed43f074c0ea&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=1&java=1&gears=0&ag=0&cookie=1&res=1366x768&gt_ms=581 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "*****", referrer: "*****"
2018/04/04 00:10:02 [error] 22749#0: *3966 FastCGI sent in stderr: "PHP message: Error in Matomo (tracker): Invalid idSite: '0'" while reading response header from upstream, client: 182.253.139.67, server: m.sudrien.net, request: "GET /js/?pg=*****&appid=5&lt=log&pv_rf=https%3A%2F%2Fwww.google.co.id%2F&uuid=e880d415-6fcd-4963-9953-ed43f074c0ea&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=1&java=1&gears=0&ag=0&cookie=1&res=1366x768&gt_ms=539 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "*****", referrer: "*****"
pablitok commented 6 years ago

Same problem here.

huang0808 commented 6 years ago

@SVyatoslavG I met this error too, Did you fix them? mysite is no idsite=0

huang0808 commented 6 years ago

@pablitok @sgiehl how did you fix it? I met too.

alekseyp commented 5 years ago

Same issue.

In my case it's trying to get appid instead of idsite and only for some users. Is it possible it's doing it for mobile inapp views thinking it's an app and not mobile page?

jimoquinn commented 5 years ago

In my case it was solved by changing idSite to idsite:

`

To: `

tsteur commented 5 years ago

I'll close this issue as this should only happen when there is a typo in the tracking code or a wrong idSite is set in the tracking code due to some issue in the user's website / app. Feel free to comment if there's any issue in Matomo.

Daijobou commented 3 years ago

@tsteur I don't think it's a typo in all cases. I have thousands of page views daily and only a handful of these messages. I rather suspect that someone has copied my page and is using it offline privately and that's why the error happens only a few handful daily. So it would be nice to be able to turn this warning off or otherwise suppress it in a meaningful way.

tsteur commented 3 years ago

@Daijobou to suppress these errors, would it work to define for example a firewall rule or an .htaccess rule to block the request if the request goes to matomo.php or piwik.php and has a URL parameter idsite=0?

tsteur commented 3 years ago

@Daijobou btw how did you embed the tracking code? If they copy / download the page then it should copy also the idsite unless you're using the tag manager and the idsite is defined through a variable.

Daijobou commented 3 years ago

If I have thousands of page views per day in matomo statistics and the "setSiteId" is set by the system in a single template file, so it can never be wrong, the problem is in my eyes somewhere else, if there are only ~ 1-15 error messages per day.

Maybe someone downloaded the page and changed the SiteId via Search&Replace or a browser plugin does something strange here.

Is it possible to make an extended log that shows the calling URL that generated the error? As long as the user does not enter the original domain name in the (windows) hosts file (localhost), it should be easier to find the source of the error. Also for people who actually have set the SiteId somewhere wrong.

tsteur commented 3 years ago

@Daijobou if you are familiar with editing files on the server you could adjust this line in core/Tracker/Request.php and basically make that line say

throw new UnexpectedWebsiteFoundException('Invalid idSite: \'' . $idSite . '\'' . ($_SERVER['HTTP_REFERER'] ?? '') );
diff --git a/core/Tracker/Request.php b/core/Tracker/Request.php
index 2dd8be5579..1fcd92a0fe 100644
--- a/core/Tracker/Request.php
+++ b/core/Tracker/Request.php
@@ -606,7 +606,7 @@ class Request
         $idSite = $this->getIdSiteUnverified();

         if ($idSite <= 0) {
-            throw new UnexpectedWebsiteFoundException('Invalid idSite: \'' . $idSite . '\'');
+            throw new UnexpectedWebsiteFoundException('Invalid idSite: \'' . $idSite . '\'' . ($_SERVER['HTTP_REFERER'] ?? '') );
         }

         // check site actually exists, should throw UnexpectedWebsiteFoundException directly
Daijobou commented 3 years ago

Oops, forgot to submit the text.

@tsteur Thanks. However, "HTTP_REFERER" does not tell on which page the error occurs, only where the user came from. ;) I have test it and added the IP to the log output. To see if the error is generated by a single page visitor or multiple page visitors.

The error_log entries are strange, because there are entries from a IPs that have many 20-100 access_log entries but like 1-2 error_log entries for wrong idSite, but what is impossible. As I wrote, we have a single template and that contains a valid idSite value.

To me it currently seems more like extremely rarely the $_GET value for idSite is lost either by a server modul itself or by Matomo. Is there any way to view the real page where the error occurs?

tsteur commented 3 years ago

only where the user came from.

For the tracking request that would be the page where the tracking request is being issued. So exactly what we're after.

Daijobou commented 3 years ago

I don't understand. HTTP_REFERER is the page you were on before. Its not the page that call idSite. Example: I coming from another website (like google.com), so the HTTP_REFERER is empty.

But to answer; All URLs from HTTP_REFERER (if not empty) have a valid idSite.

tsteur commented 3 years ago

I don't understand. HTTP_REFERER is the page you were on before.

@Daijobou That's only for the main document. The tracking requests (and other similar requests) the HTTP Referrer is then the page that is issuing this request. AKA the page you are viewing currently. This means the above code should tell you all the pages that have idsite=0 embedded. I see you mention that on these pages they are non-empty.

Is your Matomo maybe hosted on a different domain? In that case you might not get the full URL as referrer because of a Referrer Policy: strict-origin-when-cross-origin header.

You could also try this:

 throw new UnexpectedWebsiteFoundException('Invalid idSite: \'' . $idSite . '\'' . ($_REQUEST['url'] ?? '') );
Daijobou commented 3 years ago

@tsteur $_REQUEST['url'] and $_REQUEST['idsite'] is always empty in logfile. So I changed to $_SERVER['REQUEST_URI'] to show the current URI. I get this lines in errorlog:

"PHP message: Error in Matomo (tracker): Invalid idSite: '0' URI=/matomo.php?undefined&pv_id=ZHe5B6
"PHP message: Error in Matomo (tracker): Invalid idSite: '0' URI=/matomo.php?pg=title&appid=2&lt=log&pv_rf=https%3A%2F%2Fwww.bing.com%2F&uuid=f9c408a0...&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=1&java=0&gears=0&ag=0&cookie=1&res=1366x768&gt_ms=333

This look like more a bug in matomo?

and since few days I get this new error message in errorlog: "PHP message: Error in Matomo (tracker): Tracker API 'region' was used, requires valid token_auth"

What is that supposed to tell me? ;) I don't use a API and nothing changed in matomo.

tsteur commented 3 years ago

@Daijobou are you using the Matomo JavaScript tracker? Sounds like this is maybe used in a mobile app or so?

Daijobou commented 3 years ago

We use the JavaScript tracker. We don't have a mobile app, only the website.

tsteur commented 3 years ago

@Daijobou these above requests don't look like they are generated from our JavaScript tracker. Maybe look at the IP addresses from which these requests originate (eg using something like https://www.iplocation.net/ ). It looks like someone is sending random tracking requests to your server.
You could also check the "user agent" for these tracking requests whether they are sending maybe some not normal user agent header that isn't a regular web browser.

Daijobou commented 3 years ago

Thats strange. I never submit my last post, because I wanted to write more about it. ;) I'm waited for new entries in errorlog, because I already added User-Agent, but entries are very rare (only 1-15 daily). Currently there is only a single new entry with User Agent:

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0"

The IPs are site visitors via Telekom Germany and other regular providers.

I checked his IP in access_log and found a single entry and this page have a valid siteid.

tsteur commented 3 years ago

Can you share with us a link to your website maybe on a page where this might happen? Generally, these requests should definitely not be generated from our JS tracker (but anything is possible)

Daijobou commented 3 years ago

Isn't the error message fundamentally wrong? The idsite is not invalid, it is missing completely :) I have searched google for pv_id and found https://developer.matomo.org/api-reference/tracking-api

Both "idsite" and "rec" are required parameters. Since these parameters are missing, the script should abort much sooner with an appropriate error message? Does it even need to save an error message in the errorlog here? Shouldn't the API just report this kind of problem in the return value?

tsteur commented 3 years ago

@Daijobou we could probably treat a missing idsite not as 0 indeed and change the message in the logs. However, the JavaScript tracker would always set this idsite so it's still not clear where or how these requests are generated.

danielpunkass commented 2 years ago

I came across this thread after years of noticing but ignoring similar errors in my PHP logs. They only happen every few days for me, but have happened consistently for years. I'll try to apply some of the clues about narrowing down the source of the problem and see if I can come up with any new ideas. @Daijobou since October have you had any other insights?

Daijobou commented 2 years ago

@danielpunkass I haven't looked at that since. Because https://github.com/matomo-org/matomo/issues/12264#issuecomment-945111551 This URLs are not generated from my website. Either Matomo does something wrong when creating the URL or someone else generates these strange URLs. Therefore I ignore the error. Is only 0 to 5 times per day anyway.

danielpunkass commented 2 years ago

@Daijobou Thanks! I'm looking into the issue now on my end. In my case the same issue seems to occur that instead of the expected parameters to the PHP script, it inserts "undefined". So it does seem like some set of circumstances in the JavaScript will lead to the problem. I'll post again if I make any discoveries.

danielpunkass commented 2 years ago

I think I have a good idea what is going on. I don't pretend to understand the inner workings of Matomo too well, but auditing the code path that would lead to the symptoms I'm seeing, I think what happens is in getRequest, when appendAvailablePerformanceMetrics is called:

https://github.com/matomo-org/matomo/blob/9ed7e1815e263aa8a1a52332311ba0fd3489bb01/js/piwik.js#L3837

In some circumstances code will be reached in that method that simply returns instead of returning the existing request. See for example the return statement here:

https://github.com/matomo-org/matomo/blob/9ed7e1815e263aa8a1a52332311ba0fd3489bb01/js/piwik.js#L3562

And in several other instances in that method.

@tsteur In these exceptional cases should it be "return request" instead of just "return" ? This would prevent the scenario where request becomes undefined and leads to the symptom both @Daijobou and I are seeing.

In case you're curious in the past 2 years it looks like I've had 43 instances of this issue occuring, so it's not exactly common. Also, it appears that 40 of the occasions have been from Safari clients, and the other 3 from Firefox.

The only reason I've noticed these is because I keep a close watch on unexpected PHP errors on my site, and this recurring Matomo issue is one of the only issues that routinely pops up for me these days.

danielpunkass commented 2 years ago

@Daijobou If you see the issue as frequently as 5 times per day, perhaps you could try replacing all the bare "return;" lines near the line I referenced with "return request;" to confirm that the issue goes away?

Seb35 commented 2 years ago

@danielpunkass @Daijobou I observed this error (POST /matomo.php?undefined&pv_id=hnuJWg) at the rate of 2-5 errors/day. I patched it two days ago ("return;" → "return request;") and there are no more errors, so I can confirm the patch works in practice.

@danielpunkass Do you want to submit the Pull Request? Else I can submit it mentioning you.

danielpunkass commented 2 years ago

Hi @Seb35 - thanks for testing it! Glad to hear it made such an obvious improvement. I'll submit a PR today.

danielpunkass commented 2 years ago

Thanks @Seb35 @Daijobou I have submitted the PR #19232. I hope it is straightforward and can be integrated soon. @tsteur please let me know if you have any questions.

Daijobou commented 1 year ago

Its late, but I wanted to mention that, at least for me, the entry "Invalid idSite: '0'" still appears in the php 8.1 "error.log" file. However, only every few days 1-2 times (very rare). Matomo version: 4.12.3

[24-Nov-2022 03:53:31] WARNING: [pool example.tld] child 2121992 said into stderr: "PHP message: [example.tld] Error in Matomo (tracker): Invalid idSite: '0'"