matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.69k stars 2.62k forks source link

[Bug] Segmenting for "Number of Visits" returns non-matching visits #22176

Open 9joshua opened 4 months ago

9joshua commented 4 months ago

What happened?

Segmentation reports for Number of Visits = 1 include visitors with more than one visit. Here the number of Unique Visitors and Visits metrics should be equivalent... Screenshot from 2024-05-01 08-23-44

Searching the Visits Log shows some visitors who had 2 visits... Screenshot from 2024-05-01 08-23-07

This issue also appears to affect Custom Reports which would be expected as the RAW data (as shown in the Visits Log) does not properly apply the number of visits condition.

What should happen?

Segmenting reports where the number of visits =1 should omit all visitors with more than 1 visit.

How can this be reproduced?

  1. Create a Segment where "Number of Visits = 1"
  2. View the visit log to find visitors with more than one visit
  3. Observe the discrepancy between Visits and Unique Visitors - these metrics should be identical

Matomo version

5.0.3

PHP version

8.1

Server operating system

Linux

What browsers are you seeing the problem on?

Chrome

Computer operating system

Ubuntu

Relevant log output

No response

Validations

sgiehl commented 4 months ago

Looking at the code this can be easily explained. We storing a visit counter with the visits. So if a visitor has three visits, the first visit has stored a 1, the second a 2, ... The segment currently directly selects by this column. So if you are looking for equals 1, it will select the first visit of all visitors. So this segment is kind of broken by design. It might return correct data when looking for e.g. greater than, but everything else might always include other visitors.

9joshua commented 4 months ago

Another customer set up a segment expecting "number of visits" "at most" "1" to return only one-time visitors.

9joshua commented 4 months ago

Trying to help a customer around this issue, but in Matomo there is no way to segment visits for visitors who only visited the site once. This is important data for marketers and others wanting to know the behaviour of one-time visitors.

The dimension as it is should read 'visit number' as the dimension only pulls data for visit # 2 when this is set to 2 for example, and is not capable of considering other future visits.

Part of the issue is that Matomo does not have visitor-scoped dimensions. Data is rather grouped by visit, leaving the relationship between visits somewhat of a mystery. I have proposed having visitor-scoped custom dimensions. Just as the profilable dimension is a built-in visit-scoped dimension, 'number of visits' could be a built-in visitor-scoped dimension. There are many other potential uses for this.