matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.89k stars 2.65k forks source link

[Bug] For same idvisit, idvisitor is different in different tables. #22587

Open AltamashShaikh opened 1 month ago

AltamashShaikh commented 1 month ago

What happened?

We received a complaint from a customer that AdvertisingConversionExport plugin exports less data when they configure an export with and without clickIds.

Screenshot from 2024-09-13 15-09-57

On debugging the queries on Cloud, we came to know that issue is due to our join logic, when report is configured to export visits with clickIds, we join on idvsit log_clickid.idvisit = log_conversion.idvisit and when its configured to export all visits we join on idvisitor log_conversion.idvisitor = log_clickid.idvisitor and hence we see the difference.

We debugged further with this query to see the impact and found a mismatch of 2057 rows.

Select count(*) from log_clickid a, log_conversion b where a.idvisit=b.idvisit and a.idvisitor!=b.idvisitor and a.server_time between '2024-08-13 00:00:00' and '2024-09-13 00:00:00';

We also ran similar query between log_visit and log_conversion table and saw a similar mismatch of 905 rows

Select count(*) from log_visit a, log_conversion b where a.idvisit=b.idvisit and a.idvisitor!=b.idvisitor and b.server_time between '2024-08-13 00:00:00' and '2024-09-13 00:00:00';

What should happen?

Idvistor should be same across all tables.

How can this be reproduced?

Check JIRA.

Matomo version

5.1.1

PHP version

No response

Server operating system

No response

What browsers are you seeing the problem on?

No response

Computer operating system

No response

Relevant log output

No response

Validations

sgiehl commented 4 weeks ago

This problem also occurs when using cookie consent. When a website has the tracker configured to require cookie consent all tracking requests sent before cookie consent was given will be sent without a visitorid (generated by the tracker). This causes the actions to be grouped to a visit on backend side. Once the cookie consent was given, Matomo tracker will generate a visitorid and store it in the cookie - to be able to identify the visitor later again. This generated visitorid always differs from the one, that was previously generated by the backend. So the first tracking request sent with the new visitorid will update such in the database. This then also results in the problems described in the issue.