For ARKit _ios_, Is there a way to set object rotation to face the Camera?

[devsherif]

I am trying to position an object in front of the camera, but cannot set its initial rotation to be looking at the camera. Is there something I am missing or I have to add an invokeMethod native method for it ?

[devsherif]

I was able to make the object rotated to look at the camera using Camera's eulerAngles . And as I found that Camera EulerAngles are not exposed to the Flutter scope, I added a native Swift method to the plugin to expose Camera EulerAngles. I applied my change on a fork , pushed it and added this pull request #119 . I hope @olexale would accept the changes and merge it with the main repository.

[RobinLbt]

Hi @devsherif , I'm trying to compute a rotation using the camera eulerAngles to make the object look at the camera but it doesn't work, would you mind sharing here your code ?

[devsherif]

Hi @RobinLbt ,

Kindly find below how you can use the method I added to get camera Euler Angles, and use it to make the object looks towards the camera. See my code comments below:

arkitController.getCameraEulerAngles().then((rotationVector) { //Here is the method you should call to get camera Euler Angles
        var hit = value.last;
        var position = hit.worldTransform.getColumn(3);
       imageNode = ARKitNode(
          name: "image",
          geometry: plane,
          eulerAngles: vector.Vector3(rotationVector.y, 0, 0), //And here is the line of code you are looking for
          position: vector.Vector3(position.x, position.y + 1.7, position.z),
        );
        arkitController.add(imageNode);
      });

Hope this will help you get your task done.

Regards,

[devsherif]

And I think as @olexale has accepted my code and merged it in the repository. I think this issue should be closed.

[RobinLbt]

@devsherif Your code only works when the object is positionned in front of the camera I'm trying to achieve the same but with objects positionned all around the user. For that, I've created a new method into ARKit (custom fork) which let me get the camera position:

     func onCameraTransform(_ result:FlutterResult){
        if let frame = sceneView.session.currentFrame {
            let res = serializeArray(frame.camera.transform.columns.3)
            result(res)
        } else {
            result(nil)

I've then use those dart functions to compute the angle for each object :

dynamic getLookRotationAsVector(
    vect.Vector3 cameraPose, vect.Vector3 positionObject}) {
  vect.Vector3 direction = cameraPose - positionObject;
  vect.Quaternion lookRotation =
      getLookRotation(direction, vect.Vector3(0, 1, 0));
 return lookRotation;
}

vect.Quaternion getLookRotation(
    vect.Vector3 forwardInWorld, vect.Vector3 desiredUpInWorld) {
  // Find the rotation between the world forward and the forward to look at.
  vect.Quaternion rotateForwardToDesiredForward =
      vect.Quaternion.fromTwoVectors(vect.Vector3(0, 0, -1), forwardInWorld);

  // Recompute upwards so that it's perpendicular to the direction
  vect.Vector3 rightInWorld = forwardInWorld.cross(desiredUpInWorld);
  desiredUpInWorld = rightInWorld.cross(forwardInWorld);

  // Find the rotation between the "up" of the rotated object, and the desired up
  vect.Vector3 newUp =
      rotateForwardToDesiredForward.rotate(vect.Vector3(0, 1, 0));
  vect.Quaternion rotateNewUpToUpwards =
      vect.Quaternion.fromTwoVectors(newUp, desiredUpInWorld);

  return rotateNewUpToUpwards * rotateForwardToDesiredForward;
}

They work for ARCore, I've tested them. The difference is that ARCore use Quarternion as rotation but ARKit use Vector3.

I've made a function to convert Quaternion to euler angles :

vect.Vector3 eulerFromQuaternion(vect.Quaternion q1) {
  var heading, attitude, bank;
  double sqw = q1.w * q1.w;
  double sqx = q1.x * q1.x;
  double sqy = q1.y * q1.y;
  double sqz = q1.z * q1.z;
  double unit = sqx +
      sqy +
      sqz +
      sqw; // if normalised is one, otherwise is correction factor
  double test = q1.x * q1.y + q1.z * q1.w;
  if (test > 0.499 * unit) {
    // singularity at north pole
    heading = 2 * math.atan2(q1.x, q1.w);
    attitude = math.pi / 2;
    bank = 0;
    return vect.Vector3(bank, heading, attitude);
  }
  if (test < -0.499 * unit) {
    // singularity at south pole
    heading = -2 * math.atan2(q1.x, q1.w);
    attitude = -math.pi / 2;
    bank = 0;
    return vect.Vector3(bank, heading, attitude);
  }
  heading =
      math.atan2(2 * q1.y * q1.w - 2 * q1.x * q1.z, sqx - sqy - sqz + sqw);
  attitude = math.asin(2 * test / unit);
  bank = math.atan2(2 * q1.x * q1.w - 2 * q1.y * q1.z, -sqx + sqy - sqz + sqw);
  return vect.Vector3(bank, heading, attitude);
}

And I then use the computed rotation for the node :

final node = ARKitNode(
      name: UniqueKey().toString(),
      geometry: plane,
      position: position, // Generic position that was used to compute the rotation
      eulerAngles: vector.Vector3(rotation.y, 0, 0), //Here is where I use the computed rotation
    );

Unfortunately, this doesn't work and the objects are simply not visible in the ARView. @olexale might have a better solution