Touch coordinate discontinuity in Android RTCView

[MikkoAtWork]

I have RTCView for video stream that has different aspect ratio than react-native application viewport

<RTCView
      onLayout={(e) => {
        touch.current.sourceLayout = e.nativeEvent.layout
      }}
      onTouchStart={(e) => touch?.current.onTouchStart(e.nativeEvent)}
      onTouchMove={(e) => touch?.current.onTouchMove(e.nativeEvent)}
      onTouchEnd={(e) => touch?.current.onTouchEnd(e.nativeEvent)}
      objectFit='contain'
      mirror={false}
      streamURL={displayStream?.toURL()}
      style={$cvdStyle}
/>

The video is android desktop streaming from remote server. I would like to know coordinates relative to desktop so that I can send input events over webrtc data channel to remote server. The problem is that on Android, the coordinates of touch event are restarted from zero on visible video element whereas on iOS the "background" and visible video element share the same coordinate system. Attached images from both iOS and Android iOS (iPhone13 real device) Android (emulator Pixel3A)

Side effect of this behaviour is that points A and B have same coordinates on Android

Expected Behavior:

Continuous touch event coordinates over the RTCView component

Observed Behavior:

Android reports different touch event coordinates for visible video and background.

Steps to reproduce the issue:

Attach touch handlers to RTCView and log coordinates.

Platform Information

• React Native Version: 0.70.8
• WebRTC Module Version: 106.0.5
• Platform OS + Version: Android, iOS

[MikkoAtWork]

In this specific application I can work around the issue by using pageX/Y instead of locationX/Y which works ok'ish as long as RTCView occupies the whole screen i.e. something like

  scaleY(touch: NativeTouchEvent): number {
    let y = touch.locationY
    if (Platform.OS === 'android') {
      y = touch.pageY
      y -= StatusBar.currentHeight
    }
    const scaled = scale(y, this.fitHeight,
      this.srcHeight, this.dstHeight)
    return scaled
  }

[saghul]

Sorry just dropping a comment to mention I have no idea :-/

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.