Cannot use Entity Gestures with Collaborative Session

[chenzhutian]

Hi, I know this issue may be a little bit out of the scope of this repo, but want to ask you for help after reading your amazing posts about RealityKit

In the official example (https://developer.apple.com/documentation/arkit/creating_a_collaborative_session), the users can only add a cube to the scene without interacting with it. I try to add gesture supports to the cubes, so that the multiple users can move or rotate the cubes (whose movements/rotations will then be synced to all paired users).

But I found that if I do not add SynchronizationService, the movements of a cube in one user's scene will not be synced to others. If I add SynchronizationService, the cube won't be able to move (with error msg CollaborativeSession [Collision] Bad paramater (SphereRadius), value = 0.000000, passed to shape creation).

This is the repo of my code: https://github.com/chenzhutian/collaboration-interaction Could you please help? Do you have any idea of fixing this issue?

[maxxfrazer]

Hi!

The error message you’re seeing is not directly causing issues - that message appears for me whenever I do a hitTest/raycast etc since 13.4. I think it’s an internal RealityKit message which has something to do with how they compute raycasts.

Had a quick look through, and you shouldn’t need the DidOutputCollaborationData callback, realitykit should be doing that work for you since you set the sync service here https://github.com/chenzhutian/collaboration-interaction/blob/1813f28c101f731f4419e88fed7042148d473cb5/CollaborativeSession/ViewController.swift#L67

See the example in this repo that it isn’t called or mentioned anywhere, and the entities sync regardless.

If you add the AnchorEntity, model etc. on just one device, but still run installGestures on both once the entity has synced then that’s most of the work. The last part is transferring the ownership before actually moving the box. This could be done by catching the touchesBegan event for your ViewController, seeing if that hits your box; if it does, then request ownership and start moving it. Or by getting the response from installGestures and add a delegate to the EntityGestureRecognizers.

I’d recommend making your own gesture instead of using installGestures too, but that may be unnecessary for your purposes.

[chenzhutian]

Thank you for you reply. I have removed the DidOutputCollaborationData now. I used to think that the problem may be due to the ownership problem. But the cube even cannot be moved by the user who adds it. Say, here is what I have:

1. Pair A and B
2. A adds a cube. B can see it
3. A tries to move the cube. But the cube cannot be moved.

[maxxfrazer]

I just cloned and tried it out:

You need to remove this: https://github.com/chenzhutian/collaboration-interaction/blob/1813f28c101f731f4419e88fed7042148d473cb5/CollaborativeSession/ViewController.swift#L110

This will tell each device to add their own cube entity, I don't think that's what you want. When the entity is moved with the EntityGestureRecognizer it moves the entity, so if each device makes their own then that won't work.

Move the code to add the Anchor + ModelEntity inside the handleTap.

After that there are a few things you could do, the simplest would probably be to have a subscriber of SceneEvents.AnchoredStateChanged running from the start and when that sees your AnchorEntity get added to the scene it installs the gestures to your ModelEntity (give it a name to check it's the one you want, just like you're currently doing on line 110). This is instead of the didAdd anchors: [ARAnchor] check.

After that you'll have to be able to transfer the ownership if you want user B to influence the movement on A.

[chenzhutian]

Hi~ Thank you for your comprehensive explanation. I find out a solution: https://github.com/chenzhutian/collaboration-interaction/blob/master/CollaborativeSession/ViewController.swift#L118

that I only add the cube for a ARAnchor that comes from the user himself.

I think it is somehow tricky, the didAdd anchors: [ARAnchor] will be invoked for all ARAnchors no matter where it comes from user A or user B. But the model attached to the anchors from user B do not need you to add again.

Next I will try to handle the ownership stuff. Thank you again!

[maxxfrazer]

Glad I could help! I'm going to go ahead and close this issue now.