🐛 Coordinate system starts from top right position

[exzos28]

What's happening?

I've tried to run the example from docs and coordinate system is incorrect, it starts from top right instead of top left as it says here

Reproduceable Code

Image №1 with 0 0 position:

const frameProcessor = useSkiaFrameProcessor((frame) => {
    'worklet'
    frame.render()
    const rect = Skia.XYWHRect(0, 0, 400, 400) // {"__typename__": "\"Rect\"", "dispose": [Function dispose], "height": 400, "setLTRB": [Function setLTRB], "setXYWH": [Function setXYWH], "width": 400, "x": 0, "y": 0}
    const paint = Skia.Paint()
    paint.setColor(Skia.Color('red'))
    frame.drawRect(rect, paint)
}, [])

<Camera
  device={device}
  style={StyleSheet.absoluteFill}
  isActive={true}
  frameProcessor={frameProcessor}
  enableFpsGraph
  format={format}
/>

Image №2 with "center" position:

const frameProcessor = useSkiaFrameProcessor((frame) => {
  'worklet'
  frame.render()

  const centerX = frame.width / 2
  const centerY = frame.height / 2
  const rect = Skia.XYWHRect(centerX, centerY, 400, 400)
  const paint = Skia.Paint()
  paint.setColor(Skia.Color('red'))
  frame.drawRect(rect, paint)
}, [])

Relevant log output

11:30:17.203 PM: [info] 📸 VisionCamera.didSetProps(_:): Updating 24 props: [onInitialized, cameraId, position, enableBufferCompression, preview, onOutputOrientationChanged, onStarted, onCodeScanned, top, right, isActive, isMirrored, onViewReady, onError, onStopped, onPreviewOrientationChanged, onPreviewStopped, enableFrameProcessor, onPreviewStarted, format, left, bottom, onAverageFpsChanged, onShutter]
11:30:17.205 PM: [info] 📸 VisionCamera.configurePreviewOrientation(_:): Updating Preview rotation: portrait...
11:30:17.206 PM: [info] 📸 VisionCamera.configureOutputOrientation(_:): Updating Outputs rotation: portrait...
11:30:17.206 PM: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
11:30:17.211 PM: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: true, outputsChanged: true, videoStabilizationChanged: true, orientationChanged: true, formatChanged: true, sidePropsChanged: true, torchChanged: true, zoomChanged: true, exposureChanged: true, audioSessionChanged: true, locationChanged: true)
11:30:17.211 PM: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Input Device...
11:30:17.212 PM: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Camera com.apple.avfoundation.avcapturedevice.built-in_video:1...
11:30:17.224 PM: [info] 📸 VisionCamera.configureDevice(configuration:): Successfully configured Input Device!
11:30:17.224 PM: [info] 📸 VisionCamera.configureOutputs(configuration:): Configuring Outputs...
11:30:17.224 PM: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Video Data output...
11:30:17.226 PM: [info] 📸 VisionCamera.configureOutputs(configuration:): AVCaptureVideoDataOutput will rotate Frames to landscapeRight...
11:30:17.226 PM: [info] 📸 VisionCamera.configurePreviewOrientation(_:): Updating Preview rotation: portrait...
11:30:17.226 PM: [info] 📸 VisionCamera.configureOutputOrientation(_:): Updating Outputs rotation: portrait...
11:30:17.226 PM: [info] 📸 VisionCamera.configureOutputs(configuration:): Successfully configured all outputs!
11:30:17.228 PM: [info] 📸 VisionCamera.setTargetOutputOrientation(_:): Setting target output orientation from device to device...
11:30:17.228 PM: [info] 📸 VisionCamera.configureFormat(configuration:device:): Configuring Format (4032x3024 | 1024x768@60.0 (ISO: 23.0..2208.0))...
11:30:17.229 PM: [info] 📸 VisionCamera.configureFormat(configuration:device:): Successfully configured Format!
11:30:17.230 PM: [info] 📸 VisionCamera.getPixelFormat(for:): Available Pixel Formats: ["420v", "420f", "BGRA", "&8v0", "&8f0", "&BGA"], finding best match... (pixelFormat="yuv", enableHdr={false}, enableBufferCompression={false})
11:30:17.230 PM: [info] 📸 VisionCamera.getPixelFormat(for:): Using PixelFormat: 420f...
11:30:17.394 PM: [info] 📸 VisionCamera.init(frame:session:): Preview Layer started previewing.
11:30:17.395 PM: [info] 📸 VisionCamera.configure(_:): Beginning AudioSession configuration...
11:30:17.395 PM: [info] 📸 VisionCamera.configureAudioSession(configuration:): Configuring Audio Session...
11:30:17.395 PM: [info] 📸 VisionCamera.configure(_:): Beginning Location Output configuration...
11:30:17.396 PM: [info] 📸 VisionCamera.configure(_:): Committed AudioSession configuration!
11:30:17.407 PM: [info] 📸 VisionCamera.configure(_:): Finished Location Output configuration!

Camera Device

{
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "sensorOrientation": "portrait",
  "hardwareLevel": "full",
  "minZoom": 1,
  "minFocusDistance": 12,
  "maxZoom": 123.75,
  "hasFlash": true,
  "name": "Back Camera",
  "minExposure": -8,
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "hasTorch": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "formats": [],
  "position": "back",
  "isMultiCam": false
}

Device

iPhone 12 (iOS 18.0)

VisionCamera Version

4.5.2

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

• [ ] I am using Expo
• [X] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[maintenance-hans[bot]]

Guten Tag, Hans here! 🍻

It looks like you have posted a valid issue regarding the coordinate system in the example. However, I recommend checking your frame processor code and ensuring that it aligns with the documentation you've linked. If the issue persists, please provide additional log output directly from Xcode, especially if there's any runtime error that might clarify the root cause.

Remember, mrousavy maintains this project in his free time, so sponsoring the project can help him prioritize issues and improve the library even further! You can support him by becoming a sponsor here.

Let’s see if we can get this sorted out!

Note: If you think I made a mistake, please ping @mrousavy to take a look.

[exzos28]

@mrousavy please have a look if you have free time

[mrousavy]

I don't have free time.

[j0na1han]

@exzos28 Do you solve this problem? I have the same.

[exzos28]

@j0na1han I don't. Try using an older version, like 4.0 - 4.1

[j0na1han]

4.0.0 working as expected. Thanks for the hint.

[ronickg]

I think the docs need to be updated to reflect these changes from this update.

For instance what i did here to center the text vertically when in portrait mode:

    frame.save()

    const text = 'Centered Text'
    const marginTop = 400

    frame.translate(frame.width / 2, frame.height / 2)

    frame.rotate(270, 0, 0)

    frame.translate(-frame.height / 2, -frame.width / 2)

    frame.drawText(text, frame.height / 2 - font.measureText(text).width / 2, font.measureText(text).height + marginTop, paint, font)

    frame.restore()

Result:

As effectively the top left corner is the corner on the top right when looking at it vertically making the top left corner when looking at it vertically 270 degrees clockwise. The update info sums it up better with the images.

My code could also maybe still be improved.

Also not the frame.orientation is now fixed meaning it won't change. The preview orientation does though.