[Remote rendering problem] Main camera matrix is not reset after disconnecting from a session in Local or Passthrough pose modes

[robbbbbb]

Describe the problem

When disconnecting from a desktop remote rendering session that is using PoseMode.Local or PoseMode.Passthrough, the Unity main camera view matrix will remain frozen despite updates to the camera transform itself.

Provide additional information

This can be repro'd in the showcase app by the following steps:

• Open the project in the Unity editor
• Modify RemoteRenderingService.RemoteRenderingMachine to default to PoseMode.Local or PoseMode.Passthrough
• Open the SampleSceneDesktop.unity and run the scene
• Connect to a session using Rendering > Start Session
• Load any model
• Note that the camera controls work and update both the camera transform and rendered image
• Disconnect using Rendering > Stop Session
• Note that the camera controls now only update the camera transform, but the camera's rendering view never changes

This is caused by code in the ARR package in Microsoft.Azure.RemoteRendering.Unity.RemoteManagerUnity. In the UpdateLocal() function the code is setting Camera.worldToCameraMatrix every frame, which is correct, however if you ever set this explicitly you need to then call Camera.ResetWorldToCameraMatrix() as per the docs. Since this never gets called, as soon as the UpdateLocal() function stops being called, the camera matrix just freezes, giving the results described.

We've fixed this in our local version of the package by adding the call to ResetWorldToCameraMatrix() to RemoteManagerUnity.OnConnectionStatusChanged() which seems to be the place where other main camera properties are being returned to correct defaults.

[jumeder]

Hi @robbbbbb, thanks for your report and your fix suggestion. I can confirm we're able to reproduce this issue. I'll update you as soon as we have updated the client sdk!

[jumeder]

Hi again @robbbbbb! I'm happy to report we fixed the issue in the current release 1.1.32.