Add support for multiple cameras

[kring]

Currently, Cesium for Unity exclusively uses Camera.main for tile selection. We need to provide users with a way to add more cameras when needed. Something like Cesium for Unreal's CameraManager is a good catch-all, but there are probably some easier/automatic things we can do as well, such as collecting all cameras tagged MainCamera.

Requested here: https://community.cesium.com/t/about-more-main-camera-in-unity/24433/4 And here: https://community.cesium.com/t/get-location-or-bounds-of-tileset/25177/5

[jonmartincom]

I found this Main Cam issue as load time seemed awful, I.e. not loading from nearest point but filling from the far back frustrum, so changed my AR cams to not be main and used the main cam for a top down view, rendering AR views of the content loaded from the dedicated main cam frustrum, it worked 10-20x faster than having it try and load in from a full horizontal type view! good call though although I think your talking about the opposite problem of only using one cameras view over many, another problem to crack I guess.

Another option that would be useful is to somehow disconect the LOD from the main cam view, i.e. being able to override, I found with the top down view and 170 deg FOV even then with height set, I was only getting High LOD in the center view and had to play around a fair bit.

[dhyams]

I have run into this issue as well, although I am a Unity newbie and might be misusing a camera. In my experience, if a second camera is added, the landscape in general does not fill in, on one or the other of the cameras. I believe that this is the same problem as described in the original posting by @kring. The below is simply two cameras in the scene with viewports so that you can see both.

I would be glad to test if there are modifications made.

[kring]

Yes that sounds like the same problem @dhyams.

[dhyams]

I was looking and trying to fix this; and it seems pretty clear that (at least at first implementation) all it would take would be to iterate through all cameras and add ViewStates to the returned array

https://github.com/CesiumGS/cesium-unity/blob/3bd1519845f92a236727c93810e92936a51fe51a/native~/Runtime/src/CameraManager.cpp#L100-L104

But, I don't see any API for getting all cameras on the C++ side, equivalent to https://docs.unity3d.com/ScriptReference/Camera-allCameras.html

I was wondering if anyone knew how to go get a list of cameras on the C++ side; I am sure I'm missing something. But I'd like to help.

[kring]

Making new Unity functionality available to C++ is easy. See here: https://cesium.com/blog/2023/01/26/developing-for-unity/

In short, you just need to add some code that uses Camera.allCameras to the ConfigureReinterop.cs file and let Unity compile it.

[dhyams]

@kring ing Thank you for that....got me going on it. Another word of advice though might really help; so far I have added the following to ExposeCPP():

Camera c = Camera.main;
Camera[] cams = Camera.allCameras; 
int ncams = Camera.allCamerasCount; 
Camera camera_get = cams[0]; // <- I thought this would help with the problem below, but didn't

But it still doesn't seem to expose any "getter" on the C++ side to be able to read an item from an array. So on the C++ side, I can't iterate through a list of cameras and get each camera from the list, like the following (some logic removed for clarity)

::DotNet::System::Array1<Camera> cameras = Camera::allCameras();
for (int i = 0; i < cameras.Length(); i++) {
     Camera camera = cameras[i]; // won't compile; tried cameras.Item() also but only was a setter not a getter
     // do something with the camera, like add it to the list of ViewStates
}

[kring]

cameras[i] is correct. Did you let Unity recompile the source code after you added Camera camera_get = cams[0];? Also make sure you have #include <DotNet/System/Array1.h> at the top.

[kring]

Also note that there are two ConfigureReinterop.cs files. Make sure you have modified the correct one (if in doubt, try adding it to both).

[dhyams]

@kring Yes, I have let Unity recompile multiple times; I will have to admit that I am not 100% sure of the right procedure though. What I have been doing is

• modify ConfigureReinterop.cs
• load a project in Unity that uses this as a package to let it recompile; close the project
• rebuild the C++ project and code (both of the cmake steps in the Developer guide)
• load the project in Unity again, and try things out.

I'm pretty sure that I modified the right ConfigureReinterop.cs file; as the changes I've made in ExposeToCPP have exposed the Camera.allCameras api. It's com.cesium.unity/Runtime/ConfigureReinterop.cs. I have not been modifying com.cesium.unity/Editor/ConfigureReinterop.cs.

I'm sure that my difficulties are probably just being a newbie and not quite understanding the architecture of the plugin yet.

[ADDENDUM]: I just added the "Camera" lines a couple of posts up to the Editor/ConfigureReinterop.cs, and suddenly I have new code in Array1.cs that looks promising...continuing to work it...

[kring]

Hmm I'm not sure then. There are other (working) examples of arrays being accessed by index, so I don't know any reason it wouldn't work in your case. Sometimes it's useful to just add a comment to ConfigureReinterop.cs and save it to force Unity to recompile. My workflow is generally to keep Unity open all the time. When you change ConfigureReinterop.cs and save, Unity will recompile and reload the assembly. You may start getting errors about the native code being mismatched, but that's ok. You should then be able to use the new functionality from C++ immediately. The only time you need to shut down Unity and restart it is when you compile the native code, because Unity is not capable of hot-reloading it.

[dhyams]

I think it's working now, thanks @kring! I'm getting tiles in all cameras now. Performance is taking a hit; to be expected, and probably the reason it was implemented as it was. I'll put together a pull request as soon as I can, but definitely don't recommend merging until there's an option to only turn this on if the developer wants. Also getting a lot of tile dropouts on all cameras, which is something that I haven't seen before on the main camera.

I basically did the naive thing and just populated the ViewStates C++ vector with the viewstates from all enabled cameras.

[kring]

Ok, I'll take a look when you open the PR. I wouldn't expect any performance impact from this approach in the case where you have just one camera, so maybe something is wrong.

[dhyams]

Sorry, I wasn't clear; no performance impact that I can see with just one camera...I had four going.

[kring]

Ok, so that will force it to load tiles of the appropriate LOD to satisfy all of the cameras. If they're looking at totally different areas it could end up being 4x slower. That's pretty unavoidable, though.

Perhaps rather than using all cameras, we should have a component that you can add to your camera in order to designate it as a camera to use for tile selection?

[dhyams]

Agreed; that would make sense, although for me, I want good tiles in every camera...it's multiple camera views watching an object moving through the world.

In the end, I think a lot of my confusion stemmed from Visual Studio not updating its evaluation of what was legal and what was not; and I was trusting the red squiggles to know if I was doing something wrong. When I started rebuilding without worrying about the squiggles, it built fine. Just put too much trust in VStudio in this case, being so unfamiliar with the code base.

[kring]

Agreed; that would make sense, although for me, I want good tiles in every camera...it's multiple camera views watching an object moving through the world.

👍

In the end, I think a lot of my confusion stemmed from Visual Studio not updating its evaluation of what was legal and what was not

I've had good luck with Visual Studio Code, FWIW.

[dhyams]

For reference, or if someone wants to pick it up and test to see what their results are. While tiles show up in all cameras now, performance is poor (for me), and there are many situations in which all tiles don't show up (holes in the terrain, etc).

https://github.com/dhyams/cesium-unity/tree/add_support_for_multiple_cameras_327

[raggnic]

thank you for this @dhyams it was very helpful

[j9liu]

Another forum case that would benefit from this: https://community.cesium.com/t/having-photorealistic-3d-tiles-content-ready-when-the-camera-arrives/31635

[kring]

This was added in #421.