Accurate User Detection cross devices: User ID (set in JS and all other clients)

[mattab]

The goal of this ticket is to allow tracking of unique visitors across devices via the use of a User ID. This is similar to Universal Analytics User ID feature.

Tasks:

• add new &uid parameter to the tracking API.
  • this parameter can be set to any string. The string will be hashed, and used as "User ID".
  • this is almost the same as &cid= except cid must be a 16 characters hex string.
  • &uid can be set to any string and Tracking API will take care of hashing it.
  • should we store both the hash, and also the raw User ID?
• add new setUserId() javascript tracking function
• add new setUserId() PHP client tracking function
  • uid can be set by default on Tracking API requests
  • cid does not require token_auth by default (since &userId does not). see #6109
• Add custom segment userId to select only actions by this User ID.

  Use cases

Let's make it work at least in following use cases, and add an automated test for it.

• User starts visit logged out. Then user logs in. The User ID is set.
• User starts visit logged in. The user ID is set.
• User starts visit logged in. The user ID is set. User logs out. User ID is not set.
• Tracking API requests when both cid and uid are used -> which one is used?
• When the User ID changes in the same session, create a new visit. + add test

  Docs & FAQ

Todo:

• Add a FAQ for this feature "How do I specify the User ID (unique ID across devices) to increase accuracy of Unique Visitors reporting?"
• Also update the JS doc at: http://piwik.org/docs/javascript-tracking/
• And the reference at: http://piwik.org/docs/tracking-api/reference/
• Also integrate other related FAQs somewhere that makes sense eg. http://piwik.org/faq/how-to/faq_190/ http://piwik.org/faq/how-to/faq_175/ and related.
• Also we should note that "Unique Visitors" means in reality "Unique Devices" and maybe this could be mentionned in Column documentation + User docs. See forum post
• Resources & inspiration, Google Analytics doc,

  Notes

• Piwik has two User IDs currently:
  • idvisitor which is set to a random value, or the one found in the first party cookie (which was random UUID), see getVisitorIdcookie()
  • config_id which is a value that is not random, and is set to a of the user attributes (user agent, IP, etc), see getConfigHash()
• See: Universal Analytics User ID
• Later we could add
  • Setting to let Piwik admin disable this feature
  • "Cross Device report"

Once feature is done, notify Android SDK project to brief on API and make sure feature will be there.

[anonymous-matomo-user]

Or instead, can we just set a custom request in the JavaScript?

For example:

piwikTracker.setExtraRequest({
    visitor_id: 'your id',
    additional_data: { /*......*/ }
});

In this case, we can capture as many parameters in the plugins as we can.

[mattab]

Cheers @tsteur for the ping

[hpvd]

I'm not sure about the way how to do it best. Should we really set/change the visitor ID for cross device detection if the user "gets a name" (e.g. by logging in)?

Maybe another way is to generated a new additional "User-ID" to which we could add all Visitor IDs.

so we have a structure like that:

User-ID

• Visitor-ID (the same user on his desktop @ home)
• Visitor-ID (the same user on his desktop @ work)
• Visitor-ID (the same user on his tablet)
• Visitor-ID (the same user on his smart phone)
• ...

Doing it that way we could keep all details of "who is who" within piwik and the plugins of forums, shops etc (see ticket #3622) only have to submit on each login to piwik the pear of visitor ID and the account/mail/name.

[hpvd]

and another thing we should take care for:

• what happens if you track a user within one piwik installation on two different websites where each have a login (identify) possibility where the user uses different user names?
• what happen if the user visit several websites with login which uses all piwik...

[anonymous-matomo-user]

this is an almost impossible task. one would have to be truly accurate need to implement a method like comscores total universe reporting. whereby an individual is given a panel box and required to each time they logon verify id then users behaviour is assigned to that profile.

in assigning a user id to a machine one can lose site of what happens if one user logs in then someone else starts to use the machine?

the simplest solution though not 100% accurate is like in a paywell environment. a user is given a login and password. one can then assign a unique string then cross refrence it against device type or user agennt.

again limitation is user needs to login. you will have your site of those logged in vs non logged in. hope the sample ratio is enough to give you desired metrics to make informed decisions. one other thing there can also be a privacy issue when one starts to collect user data.(see googles new ad impression user data collection privacy policies)

i see the goal mentioned above but i have to ask what does this goal accomplish? what is the end result desired with the data mined from this? i think determining what this data will provide from an insights perspective should help direct this ticket.

[mattab]

See also #2211 piwik.js: Cross domain tracking

[mattab]

By default Piwik will still work as it has.

So this feature will be used only by some people. When the user is logged in on the site or app, and the webmaster knows an attribute for this particular user that they can use as "unique identifier" for them. for example could use username, uuid, email address, etc.

This will bring much more accuracy to unique users detection and tracking over time. This is simple idea, yet a powerful feature!

Sometimes called Unique User tracking, Unique devices tracking.

[mattab]

microsite.omniture.com/t2/help/en_US/sc/implement/index.html#CrossDevice_Visitor_Identification

[mattab]

Test this code http://forum.piwik.org/read.php?2,113975 and check it works fine.

[anonymous-matomo-user]

is it possible to link piwik with website login page so we can track usage by user name instead of by IP address? In other words we are using Piwik to monitor an agents usage of a website, and tracking by IP address is not as accurate as want.

[mattab]

@db394r this is exactly the goal of this ticket! stay tuned

[mattab]

I've been thinking about this and updated ticket description.

[anonymous-matomo-user]

Replying to matt:

@db394r this is exactly the goal of this ticket! stay tuned

Is there any update on this feature. It is getting really hard for me to produce correct reports for my upper management. They are wanting me to look for other options for tracking the usage of our internal website.

Thank you for your response, Daniel

[mattab]

@Daniel maybe your upper management would consider sponsoring our time to work on this feature? it would help get it out quicker for sure.

Please contact us via: http://piwik.org/consulting/ if you can help!

[hpvd]

Using this great feature, there should be a possibility to still know from where/which device which sites have been visited /which actions (events) have been made by the visitor.

Maybe one could give in the visitor profile:

• the number of devices used by this visitor and
• something like a filter e.g. with tabs to see "all visits", "visits from device 1", "visits from device 2" ....

Probably this is in details something to be implemented later but I could imagine that this is important for the technical design of this feature.

[arunvishnun]

Hi

Good to hear that this feature is being implemented in piwik. Till the time it becomes available in piwik, is there a way to at least group the current visitor log based on username ? I use custom variable to track username. Appreciate any suggestions/help in this.

Thanks Arun

[mattab]

Probably this is in details something to be implemented later but I could imagine that this is important for the technical design of this feature.

@hpvd sure that's a great idea to extend Visitor Profile to show info of configurations across visits. could you create an issue for this idea and then we could hopefully schedule it!

[hpvd]

as proposed a new issue was opened: "visitorprofile: show detailled info about device usage" #6161

[mattab]

To do before closing this ticket: create User ID user guide and FAQ and update resources.

[mattab]

New: User ID - JavaScript Tracking

[mattab]

New: User ID User guide

[mattab]

New: How do I get the number of active unique users to my app?

Updated: http://piwik.org/faq/how-to/faq_190/

Updated: How do I create new visit on demand, so that several users using the same computer or browser are counted separately?

How do I tell Piwik to track Unique Visitors based on the Visitor ID cookie, instead of using IP Address?

How do I track visits on my intranet?

[mattab]

feature is done! Check out the User ID user guide

[dmitrizzle]

Is there a way Piwik can merge visitorIDs? For example, if a user is identified with VID 1 on desktop and VID 2 on mobile and later on our system realizes it's the same guy (they've logged in with the same account on both devices).

Thanks!!

[tsteur]

I think this user feature does this

[mattab]

Use User ID and set the same User ID for a given user: http://piwik.org/docs/user-id/

[shivateja-madipalli]

I have a tricky thing to do, I'll explain in a scenario, Please guide me with this. Is there a way to get the current visitor (to my website) details from Piwik, by details I mean, IP + Referral + OS etc.. I can get all of these details from "Live.getVisitorProfile" by passing &visitorId = "xxxxxx" value.

I can also get the visitorId with "_paq.push([ function() { visitor_id = this.getVisitorId(); }])" OR "visitor_info = Piwik.getAsyncTracker().getVisitorInfo();" but these are giving entirely different visitorID for every visit from same IP address / eco system.

I just want to know how exactly is Piwik getting the value of a Visitor??

How can I get the Visitor Profile's ID? the ID shown in the image under visitors profile i.e., for every New / returning visitor Piwik is able to club visits from same IP under one umbrella, thus, there is being a tracking going on for sure, just want to know HOW??

[ryanmccoy]

@shivateja-madipalli almost a year late but just in case someone else is wondering, you can fetch the visitor ID from the tracking API (http://developer.piwik.org/api-reference/tracking-javascript)

see getVisitorId() function

[mattab]

@ryanmccoy @shivateja-madipalli this ticket is closed, could you create a new issue with the steps to reproduce the issue and what you got VS what you expected?