TagManager (variable) values are not reset for tracking, even if they are set to "" or null.

[peterbo]

In an SPA, I'm tracking a pageView with a configuration using a bunch of custom dimensions. (Virtual) PageViews are triggered by a custom event, that is pushed into the dataLayer with a bunch of variables.

The problem is, that after setting a variable, e.g. "section" to a value (e.g. "test") in the first dataLayer push, and in a later push, you set it to "" or null, the triggered pageView will nevertheless send "test" as a custom dimension value.

The Debugger correctly shows the variable as empty (""), but the issued requests contains the older value. It seems like the variable is not overwritten by the empty value in the tracker.

Step1: Variable is defined with "TEST1234567" and gets correctly included into the request.

Step2: Variable is defined with "" (empty string), but old value "TEST1234567" gets included into the request.

[tsteur]

@peterbo not 100% sure if I can follow. Do I get it right that the problem is happening when using the Matomo Tag?

I'm thinking maybe dimension.value check in https://github.com/matomo-org/tag-manager/blob/4.x-dev/Template/Tag/MatomoTag.web.js#L220 is the issue. And we should only check if the property exists but not if the value is empty/not empty

[peterbo]

@tsteur exactly!

It's an out-of-the-box Matomo Configuration containing the Custom Dimensions and an out-of-the-box Matomo PageView Tag, triggered by a certain "customPageView" event that gets pushed to the dataLayer alongside the mentioned variable. The snippet you provided is spot on. This causes the exact issue that I'm seeing!

[tsteur]

👍 great, thanks. should be an easy fix

[peterbo]

Hey everyone, sorry, we'll have to reopen this issue, since the fix introduced some unexpected / unwanted behaviour that I didn't think about before.

When e.g. setting Custom Dimensions from TagManager Variables (e.g. URL Parameter) for a visit scope custom Dimension (e.g. a click ID, Sub-ID or something), the dimension gets set on a page where the parameter is present, but gets reset (to empty) again on the following page, where the parameter is not present.

[AltamashShaikh]

@peterbo This is ought to happen with the current change. Currently we check if the object has value(not checking if its empty or null) and set the CustomDimension, earlier we were checking if the value is not empty and then only setting the customDimension.

What we can do here is go back to old solution to set only for non empty values and also add an exception to set if value is set as null

Eg: changing https://github.com/matomo-org/tag-manager/blob/4.x-dev/Template/Tag/MatomoTag.web.js#L226 to

if (dimension && TagManager.utils.isObject(dimension) && dimension.index && (dimension.value || dimension.value === null)) {
          tracker.setCustomDimension(dimension.index, dimension.value);
 }

Will it be okay if you have to set the customDimension as null when you want to reset it ?

[peterbo]

Hey @AltamashShaikh - thank you for that proposal! I'll test it in the next few days, if there is any unexpected behaviour with this!

[AltamashShaikh]

@peterbo any update on this ?

[peterbo]

@AltamashShaikh sorry for the late response!! That works well for the use-cases, that I could think of (and tested it)!

[MatomoForumNotifications]

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/tag-manager-unable-to-reset-custom-dimension-to-null-from-js-variable/53729/2

[cataclyst]

Sorry to revive this, but I think the following constellation is still a problem in this regard (as described in this forum thread as well):

Suppose I have a MTM variable of type "Custom JavaScript" that can return either a non-null value or null:

function () { return (Math.random() > 0.5) ? "SET" : null; }

Every MTM variable always has a mandatory default value mapping of null --> "" (empty string) (see here), so this MTM variable can effectively return "SET" or "".

When I use this variable as the value for an action-scoped Custom Dimension (mapped in the configuration variable), then the (second) fix introduced for this issue does not work:

if (dimension && TagManager.utils.isObject(dimension) && dimension.index && (dimension.value || dimension.value === null)) {
    tracker.setCustomDimension(dimension.index, dimension.value);
}

https://github.com/matomo-org/tag-manager/pull/593/files#diff-6073ce2936b4787bf1842db67b3199068b10ca908c39453b25e825da6d344fc8R232

because "" is neither "truthy" (to satisfy if (dimension.value)) nor null (to satisfy if (dimension.value === null)).

Also, since I only use MTM and that MTM variable is the one thing that sets my Custom Dimension, I don't have the workaround option "to set the customDimension as null when you want to reset it", as proposed in the comment above.

Basically, the default value mapping for the MTM variable is the culprit here, as I can't disable that. The original fix would have worked for this situation, but I can see how that might have a couple of unintended side effects. Is there any way that I can make this work?

[AltamashShaikh]

@cataclyst If you set the defaultValue to null for the above JS variable, does it works ?

[cataclyst]

@AltamashShaikh Hmm, yes and no. It allows the Custom Dimension to go back to the "other value", but since Matomo interprets the default value as a string ("null"), not as the special value null, that value will now show up in my reports. I could have entered "unset" or even "cake" and the effect would have been the same. 😄

That is especially unfortunate because my actual Custom Dimension is supposed to be blank 99.99% of the time. Now (almost) every action any visitor makes will track as some unimportant default value and clutter that report. I was thinking about that workaround before but dismissed it because it really generates a lot of noise and was hoping there would be another solution.

[AltamashShaikh]

@cataclyst We will discuss this issue and update you as going back to this change - https://github.com/matomo-org/tag-manager/pull/441/files seems a viable option but that will create issue for others, we might have to make this optional or a way to override it.

[AltamashShaikh]

@AltamashShaikh Hmm, yes and no. It allows the Custom Dimension to go back to the "other value", but since Matomo interprets the default value as a string ("null"), not as the special value null, that value will now show up in my reports. I could have entered "unset" or even "cake" and the effect would have been the same. 😄

That is especially unfortunate because my actual Custom Dimension is supposed to be blank 99.99% of the time. Now (almost) every action any visitor makes will track as some unimportant default value and clutter that report. I was thinking about that workaround before but dismissed it because it really generates a lot of noise and was hoping there would be another solution.

@cataclyst Can you check if this PR solves the issue for you ? String "null" would be converted to null if set for default value

[cataclyst]

@AltamashShaikh Thank you for the possible fix. Unfortunately, that doesn't work "further down": If we return null as the default value the variable, that is again converted to the string "null" in setCustomDimension(): https://github.com/matomo-org/matomo/blob/4.x-dev/js/piwik.js#L5465:

if (!isString(value)) {
    value = String(value);
}

and the effect is the same as before.

I also tried using undefined instead of null for the default value - that way it would use the check just before the one above in setCustomDimension(): https://github.com/matomo-org/matomo/blob/4.x-dev/js/piwik.js#L5462

if (!isDefined(value)) {
    value = '';
}

Buuut, in that case we are coming full circle, having the empty string "" as the value of the variable which again prevents the new value for the CD to be set in the tracker, as described in my comment above:

if (dimension && TagManager.utils.isObject(dimension) && dimension.index && (dimension.value || dimension.value === null)) {
    tracker.setCustomDimension(dimension.index, dimension.value);
}

https://github.com/matomo-org/tag-manager/pull/593/files#diff-6073ce2936b4787bf1842db67b3199068b10ca908c39453b25e825da6d344fc8R232 because "" is neither "truthy" (to satisfy if (dimension.value)) nor null (to satisfy if (dimension.value === null)).

[AltamashShaikh]

u for the possible fix. Unfortuna

@cataclyst If we set the value as null here if value is "null" does it work?

[cataclyst]

@AltamashShaikh You mean if I expanded that snippet like this?

this.setCustomDimension = function (customDimensionId, value) {
    customDimensionId = parseInt(customDimensionId, 10);
    if (customDimensionId > 0) {
        if (!isDefined(value)) {
            value = '';
        }
        if (!isString(value)) {
            value = String(value);
        }
        if (value === 'null') {
            value = null;
        }
        customDimensions[customDimensionId] = value;
    }
};

Unfortunately not. I would have expected this to work as well, and it sets the value at the correct index to null in the Custom Dimensions object. But the reports in Matomo show this as the string value "null", nonetheless. I even tried undefined instead of null again, and now a new row for the value "undefined" shows up in the reports.

I guess that somewhere, whatever key the value for a key is always gets converted to a string (even null => "null" and undefined => "undefined"), so that shows up in the reports. I think it needs a deleteCustomDimension() for this case instead?

[AltamashShaikh]

@cataclyst Thank's for checking this, for now I think having null and set could be the only solution and we might need to think about other fixes, I have added this issue for-prioritisation so that our Product team can check and schedule this one

[cataclyst]

Thanks for the update @AltamashShaikh. I appreciate you taking this back to the team - it really seems like a tough nut to crack. Since there doesn't seem to be an easy workaround, I will discuss with my client if we should accept having the "noise" in the reports (such as a "null" row with a really high count), as long as the rest of the data is correct.