matomo-org / matomo-log-analytics

Import any kind of server logs in Matomo for powerful log analytics. Universal log file parsing and reporting.
https://matomo.org/log-analytics/
GNU General Public License v3.0
224 stars 118 forks source link

Subdomain listed as "Page url not defined" in Visits Log #365

Closed L3on1d closed 7 months ago

L3on1d commented 7 months ago

Hi!

We have faced strange issue in Matomo, new subdomain listed as "Page url not defined" in Visits Log, due to we unable to create new segment based on this URL and collect statistics.

Subdomain correctly listed in Matomo website configuration (URL section).

Payload example:

image

Matomo version: 4.15.1

Thank you!

mneudert commented 7 months ago

Hi @L3on1d,

A Page URL not found should only happen if the id of the URL in the database has somehow lost its connection the string representation, or was not stored at all.

I have seen an issue you raised some time ago (matomo-org/matomo#20812) where you posted screenshots. In the one from May 31 there is already one of those "URL not found" messages displayed. And in the one from June 1 the values in your database look like you have somehow reached the max integer value (4294967295) for some action id fields.

All of those fields should be stored as INT unsigned, looking at the idaction_name column being bigger than that value, have you somehow changed your database to accommodate bigger values?

To find out if the connection is lost, there is some database querying possible (this may also answer your question in the other issue):

  1. Track a visit using a referrer that was never used before so no unrelated rows are showing up
  2. Find the visit in the log_visit table:

    SELECT * FROM log_visit WHERE referrer_url = [your unique referrer used for tracking];

  3. Find all the actions for that visit in the log_action table:

    SELECT * FROM log_link_visit_action WHERE idvisit = [idvisit value from the log_visit table];

  4. The values from the various idaction_* column should have a matching entry in the log_action table, For example every idaction_url should have a matching idaction = [idaction_url] entry. It could potentially also be a zero (0) without any action connection, those are also displayed as "not found".

If the idaction_url has no connection anymore (or is zero) it would display like this in the visits log ("Local Test" is a valid page title):

Screenshot from 2023-12-07 22-46-23

Can you check if your database has valid idaction_url connections or zeroes for one of those problematic visits?


On an somewhat related note: You have opened this issue for the log importer script and your screenshot looks like regular browser network information. Are you using this script to track the visits from your webservers logfiles?

Depending on what your database entries look like this info may could to pinpoint the source of the problems.

L3on1d commented 7 months ago

Hi! @mneudert

Thank you for your answer.

We have changed INT to BIGINT for column 'idaction' (table log_action) regarding to this issue which we faced with log importer script - https://github.com/matomo-org/matomo/issues/19386

Current issue with "Page url not defined" and https://github.com/matomo-org/matomo/issues/20812 related to js tracker, our website data is separated by segments of visits (for example by custom dimensions).

L3on1d commented 7 months ago

@mneudert any new url in DB is listed with same ID "4294967295". Old url's which was tracked correctly for many years have their own uniq ID.

image

How I understand 4294967295 is the largest INT that MySQL can hold, so transaction IDs higher than that are getting stored as 4294967295.

What we should do in our case? Change columns types to bigint?

Thank you.

UPD

After I changed related columns to bignit, I see new subdomain name in Visits log.

mneudert commented 7 months ago

Happy to hear everything is tracking again :)