Cursor/Mouse API

[crutchcorn]

Summary

Hi! I'd love to see a IMouseProps logic very similar to the existing IKeyboardProps or some other way to gain access to a mouse's location.

While this API could differ in many ways, one idea is to add props, like IKeyboardProps such as:

onMouseUp
onMouseDown
onMouseMove
onMouseOver
onMouseOut
onMouseEnter
onMouseLeave

Which maps to a very similar API to the web's API

Motivation

I'm working on a comprehensive UI component library for React Native that supports RNW.

As part of this UI component library, I'd love to have access to a mouse's cursor location in order to do the "Smart Triangle" calculation that's so common for right-clicked menus, such as this:

https://user-images.githubusercontent.com/9100169/152098909-84f302eb-a95f-4ea2-8522-3edbf5f8a504.mp4

Credit: https://height.app/blog/guide-to-build-context-menus

I'm certain there are many other candidates for UI experiences that can only be done with a mouse API of some kind

Basic Example

const App = class App extends React.Component {
  state = {
    screenX,
    screenY
  }

  private _onMouseMove = (event: IMouseEvent) => {
    this.setState({ screenX: event.nativeEvent.screenX, screenY: event.nativeEvent.screenY });
  };

  render() {
    return (
      <View onMouseMove={this._onMouseMove} />
    );
  }
}

export default App;

Open Questions

Is this unique to Windows?

It may not be, since Android and iOS also seem to have cursors in some capacity as well

[crutchcorn]

Apologies, I didn't realize this at the time but:

onResponderMove and friends seem to solve this problem for me.

[crutchcorn]

Actually, apologies, onResponderMove doesn't seem to solve my use-case. It only seems to work when a mouse click is made.

I, on the other hand, am looking for a onMouseMove style event where the user need not click in order to gain pointer access

[chrisglein]

Mouse hover support is under-documented today (tracked by https://github.com/microsoft/react-native-windows-samples/issues/541). Can you look at the discussion there and in linked issues and see if your needs are met? If not, go ahead and reopen your issue.

[crutchcorn]

Admittedly, I am unsure whether this solves my problem or not due to the undocumented nature of things currently.

However, after reading through the threads and reading through the @react-native-windows/tester package source I believe this does not solve my problem

In particular, while my outlined onMouseEnter and onMouseLeave APIs are present, my UI library's needs requires an onMouseMove event as well, which does not seem to be present.

[chrisglein]

@dedavis6797 can you take a look at this and incorporate any requests into your future spec?

[jonthysell]

@dedavis6797 Do you think this will make 0.69 or should it be moved to 0.70?

[Victor0814gui]

onMouseMove do you have an estimate of when it will be implemented?

[chrisglein]

There's these:

• Pressable.onHoverIn https://reactnative.dev/docs/pressable#onhoverin
• Pressable.onHoverOut https://reactnative.dev/docs/pressable#onhoverout

There's enough unity between the various RN platforms at this point that we'd prefer something here for all platforms, not just Windows.

I was curious how react-native-web does this and found this: https://necolas.github.io/react-native-web/docs/interactions/#responderevent-props-api I've never personally tried PanResponder, but you can take a look and see if it matches your needs?

You can also see where things might be going for alignment between React DOM and React Native with this thread: https://github.com/react-native-community/discussions-and-proposals/pull/496

[Victor0814gui]

@chrisglein, PanResponder does not meet my use case. The input and output events of both return the data I need, but neither onHoverIn nor onHoverOut operate by capturing mouse movements in the component, it has the same properties as onMouseEnter and onMouseLeave, which return the following data

{
  altKey: false
  button: -1
  ctrlKey: false
  force: 0.5
  identifier: 0
  isBarrelButtonPressed: false
  isEraser: false
  isHorizontalScrollWheel: false
  isLeftButton: false
  isMiddleButton: false
  isRightButton: false
  locationX: 698,998
  locationY: 2.98119
  pageX: 698,998
  pageY: 2.98119
  pointerType: "Mouse"
  shiftKey: false
  target: 47
  timestamp: 342842796
}

but I need events that capture the synchronous movement of the mouse, I have tried some native implementations, but without success, since implementing this with synchronous methods could make it blocking

[chrisglein]

For reference, the ways that the native stack accomplishes things like smooth scrolling that track the finger is by doing them on the composition thread itself. Even for the native controls, they avoid calling back up to the UI thread. To get the best experience here, you wouldn't want the many-layered call back up to JavaScript. You can read more about how that's done here: https://learn.microsoft.com/en-us/windows/uwp/composition/input-driven-animations Not saying a mouse position event is untenable, but there would be inherent latency and it'd never by synchronous.

[Victor0814gui]

the Button element of react-native-xaml always triggers new events when moving the mouse but almost all of them are null, through the onPointerMoved method, it's not much but it already solves a lot of problems, mainly in video players