search

matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.66k stars 2.62k forks source link

Single visits broken into multiple visits in 3.4 #12725

Open metalocator opened 6 years ago

metalocator commented 6 years ago

Hi,

We are tracking twice the visits in Matomo 3.4 v.s. 2.16.5.

We’re running both servers (3.4 v.s. 2.16.5) side-by-side and our expectation would be that they would track the same numbers in all cases.

We created the 3.4 server by duplicating our existing 2.16.5 server and running a manual upgrade. The config is the same, (minus the trusted/allowed hosts setting).

We’re funneling data from our customer Web sites to both servers by using the dual tracker method described here: https://developer.matomo.org/guides/tracking-javascript-guide#multiple-piwik-trackers

I looked at the apache log files and both servers show the same number of log entries for the same date range.

This picture of the visitor log shows it most concisely.

same_user

Notice their visit is cut arbitrarily into two visits.

That exact same user, tracking in our 2.16.5 instance is shown as a single visit.

2 16 5

The first screenshot shows more actions due to the Content impressions we track are shown as actions in 3.4, which is great.

The total visit count is roughly doubled visit_compare

We are also tracking this same site in Google Analytics, and the visit count agrees with the 2.16.5 numbers.

We can't upgrade to 3.4 until we resolve this, so any feedback, or even a hint of where to look, would be sincerely appreciated. Beyond appreciation, we are also willing to pay for assistance.

mattab commented 5 years ago

Hi @metalocator Could you please check with our latest 3.6.0-b2 beta if this issue may be fixed for you? see http://piwik.org/faq/how-to-update/faq_159/

fatica commented 5 years ago

Hi,

We saw this bug while running multiple trackers in parallel in production in a testing preparation for a cut over from 2.16.5 to 3.4.

We found that running the QueuedTracking/Redis plugin solved the problem and have been using that ever since.

Unfortunately it would be very difficult to orchestrate a test since we don't maintain parallel environments like this. Additionally we only saw the issue for our sites under medium to heavy load and I'm not sure how to simulate that. Basically when many requests would hit the log "simultaneously" according to apache log granularity.

In theory. to reproduce the issue, set up 3 environments. 2.16.5, 3.4 and the new beta. Throw the exact same high load at them quickly and see if 3.4 shows fewer visits than 2.16.5 and the new beta.

Thanks for looking into it, and sorry I can't be of more help testing.