search

googlevr / gvr-ios-sdk

Google VR SDK for iOS
http://developers.google.com/vr/ios/
Other
645 stars 191 forks source link

Trying to update head position/rotation using ARKit with Resonance Audio #332

Open BrennerHMC2 opened 5 years ago

BrennerHMC2 commented 5 years ago

I am trying to use resonance audio with ARKit. I am using version 1.140.0 for for GVRKit, since GVRAudioSDK does not work unless the version numbers align. I am having trouble getting sound sources to sound like they are coming from the correct location, e.g. I will put something in front of me in the AR space and it will sound as if it is to my right. The following code is being executed in the function :

func session(_ session: ARSession, didUpdate frame: ARFrame)

This is called every frame in order to update important information about the AR environment. 

        let angles = frame.camera.transform
        let rotation_quaternion = simd_quatf.init(angles)
        audioEngine.setHeadPosition(frame.camera.transform.columns.3.x, y: frame.camera.transform.columns.3.y, z: frame.camera.transform.columns.3.z)
        audioEngine.setHeadRotation(rotation_quaternion.vector[0], y: rotation_quaternion.vector[1], z: rotation_quaternion.vector[2], w: rotation_quaternion.angle)
        audioEngine.update()

The sound objects are being placed with the following code:

                self.audioEngine.setSoundObjectPosition(soundObjectID, x:anchor[0],y:anchor[1],z:anchor[2])
                self.audioEngine.playSound(soundObjectID, loopingEnabled: false)

where the coordinates are given by anchor.transform.columns.3.x, etc. I am asking about this here because I was having trouble finding documentation about the specifics of the coordinate system that the GvrAudioEngine uses. Is it the same as the coordinate system used in ARKit, or is there some sort of adjustment that needs to be done to make it work?