3d object appears in game view but but in stand alone application

[quangthuchien]

I’m attempting to to build my own Unity scene. I followed the read me file and have everything working correctly until I build a PC stand alone. My scene is very simple including only a cube that is supposed to displayed based on user view. In the game view, I moved in front of the kinect and the cube reacts to it. However, in the standalone version, the whole screen is black. It seems like I’m missing something obvious but I just couldn’t figure it out. Please help!

[NPatch]

Are you sure you've done everything in the tutorial, in particular this point about the Screen Setup? It's the first thing that comes to mind when we're talking differences between Editor and Standalone.

[quangthuchien]

Hey! Thank you for responding. I did set it up as multi display. I tried to run the example and also got weird result. For the example, the game view shows both the virtual texture and the virtual object but the build only shows the virtual object. And they look different Game view: Standalone build:

For my scene, after adding a virtual texture, Scene view: Game view: Standalone build:

I even created a completely new unity scene from the beginning to make sure I didn't accidentally miss any steps but I got the same result. I'm so confused.

[quangthuchien]

I don't know if this matter but the only thing I change from the original script because of an issue and solution explained here: https://github.com/Microsoft/RoomAliveToolkit/issues/75

[NPatch]

Nah these are just so you don't include editor code inside the build. If anything it should remove issues. These look like someone changes the culling masks. Also if you want to show the background, you have to set the obj hierarchy to StaticSurfaces and play around with the first ProjectionPass which is the static surfaces.I don't remember the specifics but I think I changed both shaders used. At the very least, I changed the user view shader which started with ReplacementColor....That's never a good idea if you want to show the textured mesh as is. As for the 3D object...I'd look into the Virtual Objects Mask in UserViewCamera. Tbh, I haven't completely understood the design of the framework. Normally, you'd have set the VirtualTextures layer in the Texture Layers setting in ProjectionManager and then set the Virtual3DObjects layer in the UserViewCamera's Virtual Objects Mask. That should have worked in the build and even more so in the Game View to show both the cube and the VT. But I've found it simpler to just add both of those layers on the Virtual Objects Mask setting as it appears like the CullingMask of a normal Camera. But then again, since you have differing results as is, in editor mode and standalone mode, it seems some part of the code changes the masks. So I'd recommend exposing the values either in the log file of the standalone or by using UI to make sure whether they are correct.

[quangthuchien]

I don't completely understand the framework either. I think you're on the right track. I did some trial and errors and found that my game view only shows what is the user see which is the virtual object mask. If I selected both virtual3DObjects and virtualTextures, both would show up. I think this is what you suggested but this causes both the object and the texture to move in respect to my head position Similarly, in my standalone, it only renders the texture layers as selected in the RAT Projection Manager. If I selected both virtual object and texture here, the standalone window would show both but they don't interact with the head position at all. So... somewhere between rendering passes, Unity chooses one camera to render for the game view and another for the standalone build... Now I just need to know what is responsible for this. Shouldn't calling render() manual work for both the game view and the standalone build?

[NPatch]

Supposedly there is a hidden camera that does the rendering. But I remember I had to change the shaders of the first projection pass, as well as play around with the flags of what to render....You could try setting both StaticSurfaces and VirtualTextures as the layers the first projection pass works on, but disable the user view dependent rendering. The way I understand projection passes is like a Blit with a Material on a specific RenderTexture. Basically, you run a fragment shader on the RenderTexture as if doing operations on 2D images. So if my assumptions are correct, you can use the first projection pass to render anything in its normal state and the second projection pass to render anything using UserViewDependent projection.

NOTE: I should probably say that I opted out of using UserViewDependent projection. It's too chaotic and in scenarios with multiple users probably not the best.

[quangthuchien]

I can get the scene work just fine without the head tracking element. But what I'm trying to build is more like an interactive installation than a game. So I kinda need the UserViewDependent projection. Thank you for all your help. I'll keep working on it and report back if I can find a solution for this

[NPatch]

Yeah...depends on the kind of interactive installation. When you have a common projection area but multiple users, there's no good rule as to how to warp the view with multiple user head poses, because likely everyone will have a different perspective of the scene. If it's a one user scenario then it's ok.

Anyhow, my point was, try and use the ProjectionPasses to do the drawing mostly. Add more if needed. If you find a correct combination for the user view and projection flags , as well as their respective shaders, for each pass' purpose, it should work(unless I'm mistaken in my interpretation of course :P).

Good luck.

[quangthuchien]

Yeah. Thank you for your suggestion. I'll see what I can do :)

[JossSun]

Hi, I'm experiencing similar issues and wonder if you have a clue yet? I'm creating renderings with shaders in ProjectionPass. it shows up nicely in game view. But once I build the standalone exe, there's no display at all. seems to me that the standalone executable did not get the data from Kinect at all. Any suggestions? P.S. I've set up the scene from scratch multiple times to rule out missing steps. Manged the multi display configuration and everything.