search

microsoft / MixedRealityCompanionKit

This is a MixedRealityToolkit style repository for code bits and components that may not run directly on Microsoft HoloLens or immersive headsets but instead pair with them to build experiences.
MIT License
594 stars 288 forks source link

Holograms sticking to camera #221

Open voonfoo opened 6 years ago

voonfoo commented 6 years ago

Hi, can u please guide me on how to use the SV on UNET network stack? I am having the same problem over and over again and it seems i cant get out of it for the issue of holograms sticking to camera. Please guide me on how to use SV with UNET network stack. Thank you :)

Unity version 5.6.2f1 The sample scene works for connecting

mengruoshan commented 6 years ago

I have the same problem too. Anyone please guide me through this it is so tough. Everything works but the holograms are just sticking to the camera :( Help :(

fieldsJacksonG commented 6 years ago

What do you mean by "The sample scene works for connecting"?

When you launch the application on the spectator view HoloLens, do you see a new SV_Player prefab instance under the UNetAnchorRoot in your Hierarchy?

You should also see a SpectatorView prefab instance under the UNetAnchorRoot.

If you look through the spectator view HoloLens after launching the application, has it joined a session?

It sounds like Unity is not detecting the spectator view HoloLens.

Please ensure the following:

  1. Your SpectatorViewManager prefab has a correct Spectator View ID (name or IP of the spectator view HoloLens)
  2. SpectatorViewManager's Server ID is either "editor" or a valid IP of a device running the app on your network.
  3. Your firewall is not blocking incoming UDP traffic. Navigate to "Control Panel\System and Security\Windows Defender Firewall", select "Advanced Settings", select "Inbound Rules", Find all instance of the Unity you are using (and any older instances of Unity that reference the same "Program" path) and double click on them. Select "Allow the Connection" under the "General" tab and "Domain", "Private", and "Public" under the "Advanced" tab.
  4. Your network is not blocking UDP broadcast packets. Many corporate wireless networks do, so you will need to use a separate router in that case.
  5. If you run the application on your spectator view HoloLens under a debugger, do you have any debug spew?
voonfoo commented 6 years ago

I do see the two SV_Player (Clone) and Spectator View under UNETAnchorRoot after i open the app for the spectator view hololens. However, when i tried to move the spectator view hololens and the camera, both SV_Player (Clone)'s transform did not change. Is that supposed to happen?

fieldsJacksonG commented 6 years ago

The transform on both of those prefabs should change when you move the camera around.

Please see if you have any debug spew in the app running on your spectator view HoloLens. This will help to see if your application is blocked anywhere.

voonfoo commented 6 years ago

Okay. May i know how do i check for any debug spew? Sorry i am just a newbie in this :(

fieldsJacksonG commented 6 years ago

When you built your UWP application in Unity, it created a Visual Studio solution. Run that solution with "F5" against "Device" (if you are connected over USB) or "Remote Machine" to the spectator view HoloLens' IP.

On the bottom of your Visual Studio sln, you should have an "Output" window. Keep an eye on that while the application is running to see if it is printing anything useful.

voonfoo commented 6 years ago

Hi, i have checked the debug spew and nothing is blocking the app. i have also closed all my firewall. Is there any more ip settings needed other than spectator view manager? For example, HTK network manager? Do we need to use that?

fieldsJacksonG commented 6 years ago

Can you copy the debug output? You should ensure that an anchor has been created (no pose data can be transferred if not!)

Is this on the sample application or your own application? You should have a UNETSharingStage and a UNetAnchorRoot in your project. Nothing should need to be changed in the UNETSharingStage.

voonfoo commented 6 years ago

remote player, analyzing start state UnityEngine.Debug:Log(Object) SpectatorView.PlayerController:Start() (at Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/PlayerController.cs:419)

AnchorEstablished for HoloLens-BIF3N was True is now True UnityEngine.Debug:LogFormat(String, Object[]) SpectatorView.PlayerController:AnchorEstablishedChanged(Boolean) (at Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/PlayerController.cs:141) SpectatorView.PlayerController:set_NetworkAnchorEstablished(Boolean) SpectatorView.PlayerController:CmdSendAnchorEstablished(Boolean) (at Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/PlayerController.cs:131) SpectatorView.PlayerController:InvokeCmdCmdSendAnchorEstablished(NetworkBehaviour, NetworkReader) UnityEngine.Networking.NetworkIdentity:UNetStaticUpdate()

remote device likes the anchor UnityEngine.Debug:Log(Object) SpectatorView.PlayerController:CmdSendAnchorEstablished(Boolean) (at Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/PlayerController.cs:134) SpectatorView.PlayerController:InvokeCmdCmdSendAnchorEstablished(NetworkBehaviour, NetworkReader) UnityEngine.Networking.NetworkIdentity:UNetStaticUpdate()

Setting saved anchor to bfb4d596-937b-4622-8f85-4c3e7dd67fde UnityEngine.Debug:Log(Object) SpectatorView.UNetAnchorManager:AnchorFoundRemotely() (at Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/UNetAnchorManager.cs:436) SpectatorView.PlayerController:CmdSendAnchorEstablished(Boolean) (at Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/PlayerController.cs:135) SpectatorView.PlayerController:InvokeCmdCmdSendAnchorEstablished(NetworkBehaviour, NetworkReader) UnityEngine.Networking.NetworkIdentity:UNetStaticUpdate()

It is my own application. I do have a UNETSharingStage and UNETAnchorRoot in my project.

fieldsJacksonG commented 6 years ago

We do have an anchor, that's good!

In your Unity project, after you press "Play", do you have any errors in your Unity output?

voonfoo commented 6 years ago

There are no errors but two warnings that look like this.

DontDestroyOnLoad only work for root GameObjects or components on root GameObjects. UnityEngine.Networking.NetworkManager:Awake()

DontDestroyOnLoad only work for root GameObjects or components on root GameObjects. UnityEngine.Networking.NetworkDiscovery:StartAsServer() SpectatorView.NetworkDiscoveryWithAnchors:StartHosting(String) (at Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/NetworkDiscoveryWithAnchors.cs:259) SpectatorView.SpectatorViewManager:Update() (at Assets/Addons/HolographicCameraRig/Scripts/SpectatorViewManager.cs:314)

fieldsJacksonG commented 6 years ago

Did you launch the spectator view compositor in Unity (SpectatorView tab, select "Compositor") Do you have active color frames in the compositor?

You may not get poses until you have valid color frames.

voonfoo commented 6 years ago

Yes i did launch compositor. I have tried the older version of spectator view which is using the sharing service and it worked perfectly. However, changing to UNET network stack, the spectator view camera transform just doesnt seems to change when i moved the camera :(

fieldsJacksonG commented 6 years ago

This is the block of code that sets the transform of your spectator view prefabs: https://github.com/Microsoft/MixedRealityCompanionKit/blob/master/SpectatorView/Samples/SharedHolograms/Assets/Addons/HolographicCameraRig/Scripts/SpectatorViewManager.cs#L246

It will block on having a valid color frame, so make sure your camera is connected and on.

You can also put a breakpoint in here to see if anything is being skipped, and see if currPos and currRot have valid data.

fieldsJacksonG commented 6 years ago

Just saw your last post. We need to make sure the SpectatorViewManager is a child of the SpectatorView prefab at runtime and the code that I linked above is hit.

If SpectatorViewManager is not a child of SpectatorView, you will need to ensure that this line was called: https://github.com/Microsoft/MixedRealityCompanionKit/blob/b8f8cf836370895f7915a7a8b459ee71ade3913a/SpectatorView/Samples/SharedHolograms/Assets/Addons/HolographicCameraRig/SV_UNET/Scripts/PlayerController.cs#L392

voonfoo commented 6 years ago

Hi, we tried to download another package and it worked perfectly!!! i think it might the addons problem. Thanks for the guide!!!

voonfoo commented 6 years ago

Can we add custom rotation to the UNETSharedhologram script?

KevGLewis commented 6 years ago

voonfoo98, can you elaborate on what you mean by downloading another package? I am facing the same issue with the hologram "sticking" to the camera. Thanks!