What can we do in JS? Exploration of capturing user interactions.

[bastianilso]

We need to make a clear overview of what is possible in terms of data capturing.

• How can we capture data about people's mouse clicks?
• How can we capture data about mouse movement?
• What possible storage backends are possible (JSON, CSV, MYSQL..)?

[bilal-62210]

We need to make a clear overview of what is possible in terms of data capturing.

• How can we capture data about people's mouse clicks?
• I think we can use the JS event : "mousedown/mouseup" that allow us to detect when the mouse button is pressed then released on an element.
• We can use also the event "Click" that is triggered after a mousedown event and following a mouseup on the same element, if the left mouse button was used.
• How can we capture data about mouse movement? For me we can use the event "mousemove", each movement of the mouse over an element triggers this event.
• Example of implementation: input onmousemove="this.value=event.clientX+':'+event.clientY" value="Mouse over me"
• What possible storage backends are possible (JSON, CSV, MYSQL..)? -it is possible to use flat files like json, csv, excel etc...
• but we can also use databases like SQL server, mysql, maria DB etc... or on the cloud like Azure database for example.
• It depends on how much data we will have and how we want it stored, for example a csv file is structured by commas and semicolons.

[bilal-62210]

These are my first hypotheses, what do you think?

[bastianilso]

Thanks @bilal-62210 ! Now we will need two things:

1. We will need to develop a format for capturing data. Here is my initial suggestion:

Here is an explanation of the columns:

• Timestamp: a time stamp in the yyyy-mm-dd hh:mm:ss.mmm format referencing the instant a given event happened.
• SessionID: some kind of unique ID, that indicates the user's session.
• DeviceID: a unique ID for the device.
• Event: Human-readable label of what the user is doing e.g. when they hover over a button, hover over a visualization, scroll with the scroll wheel, click with the mouse.
• Element: ID of the element the user is interacting with.
• screen-x/y: we need coordinates indicating where on the screen the user pressed (i dont know which coordinates are most suitable, but it should ideally be screen coordinates, so we can compare/visualize the data on top of e.g. a video recording).
• sectionID: this is some kind of indication of where we are on the web dashboard. (For example the "player overview" section in the sidebar). Maybe it could also make sense to have a "URL" column?
• sessionDuration: how much time has elapsed so far in the session.

1. I would like you to add to this git repository an example implementation of capturing data from javascript and save the data to CSV - for now I think we should focus on doing it purely in Javascript outside R. It could be for a basic HTML website with one or two elements. The screenshot above is your guideline for how we ideally want the data to be formated. But if you have ideas/suggestions for this you're welcome to add them! 🙂

[bastianilso]

Here is the CSV file that I screenshotted above, by the way. 🙂 example.csv

Also, in case you discover a JS library which can already capture this data and save you time, let me know! Then we can evaluate whether it makes sense to use it, or it's better to develop this data capturing system "from scratch" so to speak.

[bilal-62210]

I've put my example implementation of capturing data from javascript and save the data to CSV on the "example_implementation" succursale. You can find the js file in the folder "mouse". For my example, fields value like "SessionID","DeviceID" and "session_duration" are fictive as we are not on the real environment. This is a demo of the implementation: 2022-09-26 08-45-41.zip

Give me your feedback @bastianilso .

[bastianilso]

Hi @bilal-62210,

Nice progress, thanks for the video. I had a look at your code.

• The resulting CSV file needs to encode a header so you have the names of the columns in row 1 of the CSV file. Maybe it is already possible to do this with the JSON2CSV() function.

• The timestamp is incorrectly formatted (should be yyyy-mm-dd hh:mm:ss.mmm), most likely there is an equivalent in JS to String.format() you can use to achieve this.

   <button id="button" onmouseover="mouselog(event)" onmouseout="mouselog(event)" onclick="mouselog(event)">click me</button>
  <div id="parent" onmouseover="mouselog(event)" onmouseout="mouselog(event)">parent
    <div id="child">child</div>
  </div>

  <textarea id="text"></textarea>
  <input type="button" onclick="text.value=''" value="Clear">
  <input type="button" id="btn" onclick="btn()" value="download csv">

• I can see that many of the HTML elements calls the mouselog(event) function. Now, imagine we have 50 HTML pages with 100 elements each. It would be a lot of work to encode these function calls everywhere. A more managable way might be if we could tie the data capture to a specific class. For example, could we code the script such that if we add the CSS class logData to an element, the element will have its interactions logged? For example <button id="button" class="logData">. Just curious if it is possible.
• Alternatively, it might be possible in Javascript to access all mouseover events from all elements. See the example JQuery code below. As I read it, something like this should apply a function to every button in the whole HTML page.

$('button').bind('mouseover mousemove', function(e) { mouselog(event); });

[bilal-62210]

@bastianilso

I've put the new example implementation of data capture javascript to csv file. With Csv header label added, new date format and mouselog(event) not called on every element only on the body of the page. This is on the "new version of the example implementation" commit. Take a look and give me your feedback. A demo of the new version: 2022-09-26 11-43-43.zip

[bastianilso]

@bilal-62210 Thanks. Please tell me a bit more - why did you settle for this technical solution, instead of the solutions I proposed in my previous reply?

Some comments:

• Headers look good.
• The date format seems to be yyyy/mm/dd hh:mm:ss.mmm rather than yyyy-mm-dd hh:mm:ss.mmm.
• For the DeviceID, I suggest we try to look into how to capture a hardware mac address from Javascript. I'm pretty sure there is API to do this or something similar from the browser.

[bilal-62210]

@bastianilso I am currently looking for the deviceID and I would like to ask you if you have any leads for research because all my research leads to this :

[bastianilso]

@bilal-62210 makes sense, have a look here - there is a description of an alternative solution + an NPM package. https://stackoverflow.com/questions/49692073/how-can-i-get-a-unique-device-id-in-javascript

[bilal-62210]

@bastianilso I found a solution: i've implemente this function: wich create an ID that will be unique for browser/machine combo. the result of this function is : Do you think it can be a good solution?

[bastianilso]

@bilal-62210 sure, it might be a good solution, the function output looks good. But it depends on what is the ID based on and if it really is unique across sessions?

[bilal-62210]

@bastianilso yes i think is unique across sessions because i've test it on chrome navigation and on private navigation and the ID changed.

[bastianilso]

@bilal-62210 what is the ID based on? how is it generated? why is the function called crypto.randomUUID ? what does the "random" part mean?

[bilal-62210]

@bastianilso The ID is based on the UUID format, the function generate a random UUID and this random UUID become the machine ID. The machine ID will be unique for the browser/machine combo. Tell me if it's clear or not.

[bastianilso]

@bilal-62210 the device ID should be unique for the device. Which means, it cannot be random. We can use other measurements to determine if the device is the same, for example user agent, IP address,.. See the answer in the stackexchange thread I linked above.

[bastianilso]

@bilal-62210 ah, I understand now what you are saying it does.. OK so on the first run, the machine ID is generated and on subsequent runs, it is loaded from local storage. Sorry, this was not clear to me from your answers..this is fine then - the most important thing is, that the device ID keeps showing up if we run the test multiple times on the same machine. This is the test you can do, to make sure it works.

[bilal-62210]

@bastianilso This is a demo : 2022-09-26 14-37-07.zip You can see that on the first run the machine ID is generate and after is loaded from local storage as you said above. You can also see that the machine id change when i run the code on private navigator but it keep the machine ID for next session. So i think it can work