Feature: mouse tracking

[marcelklehr]

I've had the chance to look at this already. This is a first attempt at implementing mouse tracking functionality, to be refined once more detailed experiment requirements are available.

It can be added to a view as follows:

const main_block = magpieViews.view_generator("forced_choice", {
  // This will use all trials specified in `data`, you can user a smaller value (for testing), but not a larger value
  trials: main_trial_new.length,
  // name should be identical to the variable name
  name: 'main_trials',
  data: main_trial_new,
  mousetracking: {
    enabled: true,
    autostart: true
  }
  // you can add custom functions at different stages through a view's life cycle
  // hook: {
  //     after_response_enabled: check_response
  // }
});

And produces an additional field in the data output per trial that looks like this:

{
  // ...
  mousetrackingPath: [
    {x: 12, y: 540, time: 12},
    {x: 130, y: 510, time: 230}
  ]
  // ...
}

• x and y are pixel coordinates inside the view container (thus independent of screen dimensions)
• time is the time in milliseconds since the start of mouse tracking (this is reset for each trial).

If autostart is set to true, tracking starts when the stimulus is presented. If it's set to false, the user may call config.mousetracking.start() in the view to start tracking the mouse until the end of the trial. This could be made available for hooks as well.

Note: Tracking mouse coordinates as implemented in this PR does not happen at regular intervals but only upon movement of the mouse.

[michael-franke]

This looks excellent.

But it might be that the magpie backend has trouble with this format:

mousetrackingPath: [ {x: 12, y: 540, time: 12}, {x: 130, y: 510, time: 230}
]

Maybe use this:

{ //… xpos: [12|130|…], ypos: [540|510|…], timepos: [12|230|…] //… }

Note: Tracking mouse coordinates as implemented in this PR does not happen at regular intervals but only upon movement of the mouse.

Can this be set to regular intervals? That’s how it’s usually done.

Thanks!!

On 28. May 2020, at 16:32, Marcel Klehr notifications@github.com wrote:

've had the chance to look at this already. This is a first attempt at implementing mouse tracking functionality, to be refined once more detailed experiment requirements are available.

It can be added to a view as follows:

const main_block = magpieViews.view_generator("forced_choice", {

// This will use all trials specified in data, you can user a smaller value (for testing), but not a larger value

trials: main_trial_new.length,

// name should be identical to the variable name

name: 'main_trials',

data: main_trial_new,

mousetracking: {

enabled: true,

autostart: true

}

// you can add custom functions at different stages through a view's life cycle

// hook: {

// after_response_enabled: check_response

// } }); And produces an additional field in the data output per trial that looks like this:

{

// ...

mousetrackingPath: [

{x: 12, y: 540, time: 12},

{x: 130, y: 510, time: 230}

]

// ... } • x and y are pixel coordinates inside the view container (thus independent of screen dimensions) • time is the time in milliseconds since the start of mouse tracking (this is reset for each trial). If autostart is set to true, tracking starts when the stimulus is presented. If it's set to false, the user may call config.mousetracking.start() in the view to start tracking the mouse until the end of the trial. This could be made available for hooks as well.

Note: Tracking mouse coordinates as implemented in this PR does not happen at regular intervals but only upon movement of the mouse.

You can view, comment on, or merge this pull request online at:

https://github.com/magpie-ea/magpie-modules/pull/75

Commit Summary

• Implement mouse tracking module • Allow starting mousetracking on custom event File Changes

• A src/magpie-mousetracking.js (27) • M src/magpie-utils.js (9) • M src/magpie-views.js (3) Patch Links:

• https://github.com/magpie-ea/magpie-modules/pull/75.patch • https://github.com/magpie-ea/magpie-modules/pull/75.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[marcelklehr]

Both are fixed now.

Trial data now looks like this:

{
    data.mousetrackingX: [...]
    data.mousetrackingY: [...]
    data.mousetrackingTime: [...]
}

And mouse positions are recorded every 50ms. (Is this a sensible default?)

[marcelklehr]

moustracking now takes an optional origin for mouse coordinates.

Additionally, I've implemented a "listen_and_decide" view (for lack of a better name) that works like this:

var main_trials = [
  {
    "item_id": 23,
    "item_type": "critical",
    "item_name": "paper-plane-kite",
    "det_target": "das",
    "det_competitor": "der",
    "question": "Ist das ein Bild von einem Papierflieger?",
    "question_file": "q_r_papierflieger.wav",
    "answer_file": "r_t_papierflieger.wav",
    "complete_answer": "Das ist tatsächlich ein Bild von einem Papierflieger.",
    "expected_response": "Papierflieger",
    "target": "Papierflieger",
    "competitor": "Drachen",
    "condition": "reliable",
    "DP": "indeed",
    "picture_target": "paperairplane.jpg",
    "picture_competitor": "kite.jpg"
  },
  /*...*/
]

magpieViews.view_generator("listen_and_decide_mousetracking", {
  trials: main_trials.length,
  name: 'main_trials',
  data: main_trials,
  audioPath: 'audio-trials/',
  imagePath: 'pictures-trials/',
  decisionEvent: 'mouseover', // could also be 'click' or something else entirely
  initialDelay: 300,
});

I have used this to build a working experiment with the example data @michael-franke sent me.

EDIT: Adding moustracking: enabled in the trial config is not necessary anymore.