ParabolInc / parabol

Free online agile retrospective meeting tool
https://www.parabol.co/
Other
1.91k stars 332 forks source link

Share mouse cursor #2191

Closed mattkrick closed 4 years ago

mattkrick commented 6 years ago

When i'm in a meeting, i want to share my mouse cursor with other folks whenever i hold down the shift key.

There are a few challenges with this:

solving for scroll position would require sharing the element that i'm on top of and the scrollTop of that element. Then, if that isn't visible for another client, we just show some down arrows like super smash bros. if you click those down arrows, it'll auto scroll to where i am.

the scroll coordinates problem is harder. imagine i'm in portrait & you're in landscape. it's not enough for you to know the %x and %y of the viewport that i'm at because you really want to know what i'm on top of. conversely, if i share the element that i'm on top of & i move 2px off of it, i should still be near that thing.for example, if i'm in between cards 2 & 3, my mouse is currently in the body, but i want to share that i'm between cards 2 and 3. my 2 & 3 might be LTR, and your 2 & 3 might be top to bottom. when i drag my mouse from the left of 2 to the right of 3, how can i transpose that to the top of 2 to the bottom of 3 for you? that's not possible, so we need to compromise somewhere. Priority number 1 is to put the cursor on top of the thing that i'm on top of.

SPEC - SERVER

Each mousemove sends the following data:

className: '123abc'
ancestorsToClimb: 5,
ancestorKey: '5442235',
elWidth,
elHeight,
elX,
elY,
x,
y,
scrollTop
  1. className is gathered from the mousemove event target element. if the element does not have a class, it climbs recursively until it finds one. This becomes the reference node.
  2. Once a reference node is found, we call node.__reactInternalIntersance$... and recursively look at the return prop until we find a nonnull key value or the absence of another return prop. Record how many times we looked into return and the key at that time. That becomes ancestorsToClimb, ancestorKey.
  3. call node.getBoundingClientRect() to get the left, top, width, height of the reference node.
  4. Get x,y from the mousemove event itself
  5. The elX, elY coords tell you where within the element the cursor is: Math.abs(x - left), Math.abs(y - top).

SPEC - CLIENT

  1. call document.getElementsByClassName(className) to get a list of the possible elements
  2. for each element, safely look into the return property ancenstorsToClimb times
  3. if the ancestorKey matches (it could be null), break. if it's never found, return & ignore the payload
  4. call node.getBoundingClientRect() to get the left, top, width, height of the reference node.
  5. If the size & aspect ratio are similar (heuristic TBD), determine coordinates by elX / elWidth, elY / elHeight. Memoize & record event in a stack
  6. If the size & aspect ratio are not similar, determine a movement diff based on the previous events (xDiff = x(t) - x(t-1)). if the cursor is within NEARBY_THRESHOLD (TBD), add the diff to the previous event to determine the location. Multiply it by an element area coefficient: myElementWidth / serverElementWidth) * xDiff + x(t-1). For example, if the server was 5 pixels away from the right border of a button & the next event shows 5 pixels to the right of that button & my screen is 90% the width of the server, this will find a diff of 10 * .9 = 9 pixels to the right of the previous location.
  7. If there is no previous event inside a fixed component, or the NEARBY_THRESHOLD is exceeded, use the naive estimation shown in Step 5.

@jordanh we're basically building a coordinate transposition engine here. any holes you can poke in this now will save a ton of time later!


use cases:

assumptions:

shortcomings:


Options considered to arrive at the spec:


effort: 13 points

acceptance criteria:

mattkrick commented 6 years ago

i think we arrived at a good heuristic in the grouping phase. we can use that same heuristic for other areas where we want to share cursor position

jordanh commented 4 years ago

I still think this is a good idea. Could we re-open this one @mattkrick ?

mattkrick commented 4 years ago

why sure!

github-actions[bot] commented 4 years ago

Stale issue

mattkrick commented 4 years ago

shuttin er down