Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
Added a config for custom_dimensions_max_length and refactor plugins/CustomDimensions/Tracker/CustomDimensionsRequestProcessor.php:prepareValue() to use the config instead of being hardcoded.
As a user, I want to track a custom dimension with a value that exceeds the limit, so that I can observe how Matomo handles it when the value length is capped at 250 characters.
Which equals 250 characters exactly.
custom_dimension_1 type is varchar(255) in the database.
Story 2:
As a user, I want to track a custom dimension that reaches the 255-character limit, so that I can verify how Matomo handles and stores the full length.
Which equals 255 characters exactly.
custom_dimension_1 type is varchar(255) in the database.
Story 3:
As a user, I want to track a custom dimension that exceeds the 255-character limit, so that I can observe Matomo’s behavior when the input length causes data storage to fail without warning.
$inputMaxLength = 256; // 400 Bad Request
Stored value in MySQL:
'' // empty string
This means that MySQL is not storing the value because it is too long. No error is shown in the UI and there is no Tracking failures in the log.
custom_dimension_1 type is varchar(255) in the database.
Story 4:
As a user, I want to track a custom dimension with a value longer than the default maximum length by adjusting the database schema to support larger values, so that I can ensure the system behaves correctly with longer data.
I need to change the default column type for custom dimensions in the database from varchar(255) to text.
According to this issue in Github: https://github.com/matomo-org/matomo/issues/16150 we can alter the column type in the database like this:
After a successful change in the database, I will track a custom dimension with a value that is too long so that I can see how Matomo behaves.
custom_dimension_1 type is now TEXT in the database.
Which equals 256 characters exactly.
custom_dimension_1 type is TEXT
Story 5:
As a user, I want to track a custom dimension with a value that approaches the TEXT field limit (65,535 characters), so that I can verify that Matomo handles maximum-length values properly.
Which equals 250 characters exactly.
custom_dimension_1 type is TEXT in the database.
Conclusion
The default maximum length for custom dimensions is 250 characters. If a value exceeds this limit, it will be truncated and stored, unless the length exceeds 255 characters (the database limit for varchar(255)), in which case it will not be stored at all. Additionally, no error is displayed in the user interface, and no tracking failures are logged.
To fix this issue, a user needs to change the column type for custom dimensions in the database from varchar(255) to TEXT.
Potential edge cases thought about:
The TEXT type is not as efficient as varchar(255) for shorter values. TEXT fields are generally less efficient than VARCHAR fields for smaller values. This can have performance implications for large datasets with frequent updates or queries on these fields.
The TEXT type can store up to 65535 characters, which is a lot more than the 250 characters needed for custom dimensions.
Switching to TEXT fields could result in increased storage use, particularly if a large number of custom dimension values exceed the original 255-character limit, leading to larger data entries.
If the custom dimension columns are indexed for performance reasons, switching to a TEXT field could limit the ability to index effectively, as MySQL has restrictions on how much of a TEXT field can be indexed. This could potentially affect query performance, especially if the dimensions are frequently used in queries.
A user needs to alter the column type in the database manually.
A user updates only the custom_dimension_1 column, but there are 5 custom dimensions in total. The user needs to update all of them.
A user only updates the custom_dimension columns in the matomo_log_visit table, but there are other tables where custom dimensions are stored.
The log_link_visit_action and log_conversion tables also have custom dimension columns that need to be updated.
It’s essential to ensure that any schema changes are consistently applied across all relevant tables.
Users might not fully understand why they would need to manually update the schema or why the field behaves differently without adjusting the database, leading to confusion. This should be well-documented in the UI or support materials to minimize misunderstandings.
[x] Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
[x] Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
Added a config for custom_dimensions_max_length and refactor plugins/CustomDimensions/Tracker/CustomDimensionsRequestProcessor.php:prepareValue() to use the config instead of being hardcoded.
Description:
I will change the max length for custom dimensions in: https://github.com/matomo-org/matomo/blob/4.11.0-rc1/plugins/CustomDimensions/Tracker/CustomDimensionsRequestProcessor.php#L175
To see how it behaves before I apply my fix.
I'm tracking custom dimensions with a default installation of Matomo. No extra configuration or plugins added.
The default
custom_dimension_1
type is varchar(255) in the database.The tracking code is:
I'm modifying the php function like this:
Story 1:
Stored value in MySQL:
Which equals 250 characters exactly. custom_dimension_1 type is varchar(255) in the database.
Story 2:
Stored value in MySQL:
Which equals 255 characters exactly. custom_dimension_1 type is varchar(255) in the database.
Story 3:
Stored value in MySQL:
This means that MySQL is not storing the value because it is too long. No error is shown in the UI and there is no Tracking failures in the log. custom_dimension_1 type is varchar(255) in the database.
Story 4:
Which would also help with another unrelated issue: https://forum.matomo.org/t/error-while-updating-to-4-0-0-b1-have-to-change-some-columns-to-text-or-blobs/39482
After a successful change in the database, I will track a custom dimension with a value that is too long so that I can see how Matomo behaves. custom_dimension_1 type is now TEXT in the database.
Stored value in MySQL:
Which equals 256 characters exactly. custom_dimension_1 type is TEXT
Story 5:
Stored value in MySQL:
Which equals 250 characters exactly. custom_dimension_1 type is TEXT in the database.
Conclusion
The default maximum length for custom dimensions is 250 characters. If a value exceeds this limit, it will be truncated and stored, unless the length exceeds 255 characters (the database limit for varchar(255)), in which case it will not be stored at all. Additionally, no error is displayed in the user interface, and no tracking failures are logged.
To fix this issue, a user needs to change the column type for custom dimensions in the database from varchar(255) to TEXT.
Potential edge cases thought about:
log_link_visit_action
andlog_conversion
tables also have custom dimension columns that need to be updated.Review