Support for VR

[StijnOosterlinck]

Hi, I was wondering if there is support for VR headsets to be used with JOAN?

[OlgerSiebinga]

Hi @StijnOosterlinck,

Yes, you can use VR headsets with JOAN. But unfortunately, there is documentation on how to set that up (yet). We use the develop branch of JOAN with Carla 0.9.13 and the VR Expansion plugin in Unreal. That should be enough to support VR headsets.

[StijnOosterlinck]

Hi @OlgerSiebinga ,

Thanks for the quick reply! I'll look into it.

I had another question; If I want people (driving the ego vehicle) to follow a specific route, what would be the most convenient way to do this? Should I add signs with directions in the map itself, or can I add direction arrows somewhere on the dashboard for example?

Thanks in advance :)

[OlgerSiebinga]

Adding signs to the map is probably the easiest way to implement it. But I'm not so sure how well that will work though (people might miss signs).

You could also add stuff in the field of view of the driver, e.g., on the dashboard. We did that with the projection of the current velocity. This is implemented in the viewport tab of the blueprint of the player pawn (should be JOANPawn if I remember correctly). However, it is quite difficult to send extra information from the CARLA client (JOAN is our case) to the CARLA sim/unreal. So you'd probably have to implement all logic (i.e., where the arrow has to point) in unreal.

[StijnOosterlinck]

I got the VR setup working now, but I was wondering whether someone already tried to implement side & rear mirrors inside the car?

This is probably quiet a difficult task, but I think it leads to a better overall experience.

@OlgerSiebinga any thoughts on this?

[OlgerSiebinga]

Yes, we have tried this but we haven't succeeded yet. In all our attempts the mirrors put such a heavy load on the computer/graphics card that the simulation slowed down substantially.

However, I have heard from somebody else that he managed to get it working (with JOAN). He used the planar reflections in the Unreal Engine. But these are known to be computationally heavy, so it might depend on your hardware if that works.

[StijnOosterlinck]

However, I have heard from somebody else that he managed to get it working (with JOAN). He used the planar reflections in the Unreal Engine. But these are known to be computationally heavy, so it might depend on your hardware if that works.

I also saw some people doing this, but I'm not that familiar with Unreal Engine. Can I do this by opening the BP_EpicAudi2 BluePrint class and set the side/rear mirrors to planar reflections there (didn't find this yet) or how do I do this?

Thanks in advance!

[OlgerSiebinga]

To be honest, I'm also not familiar with this. I'll ask around and see if I can get some instructions for how to do this.

[DennizGoren]

Hi there! Adding mirrors is indeed something that can be achieved with Planar Reflections!

The first thing that should be done is adjusting the following settings in the Project Settings of Unreal Engine: The "Support global clip plane for Planar Reflections" should be checked and "Motion Blur" should be unchecked, as it can impact the rendering performance.

Afterwards the Planar Reflections can be added to the mesh of your vehicle by using the Add Component button in the Blueprint Viewport. Getting the sizes and angles right might take some patience, but it's achievable.

In my experience the extra rendering workload added by the mirrors does significantly impact the performance in VR, especially when multiple vehicles are involved. However, lowering the Engine Scalability Settings from Epic to High or Medium and using maps with fewer objects make a big difference in performance!

I hope these steps provide some help and please feel free to contact me in case you run into a problem!

[OlgerSiebinga]

great, thanks @DennizGoren!

[StijnOosterlinck]

Thanks @DennizGoren , will look into this! :)

[StijnOosterlinck]

@DennizGoren ,

I'm trying to set up the planar reflection for the middle/rear mirror.

I think I got it in the right position, but I do not know how to change the size of the plane? Adapting the scale seems to have no effect on this, and since the plane is quiet big now its difficult to set a good rotation.

Do you know where I can change the size to fit it in the mirror?

Thanks in advance!

[DennizGoren]

That is strange! Unfortunately I don't know how to help you in this case. For me, adjusting the scale did in fact change the size of the planes. The first thing I did after adding the planes in the BP, was changing scale to get them close to the size of what I want them to be. Only after getting the size somewhat right, I placed them at the right positions and adjusted the angles. I hope you manage to get it working!

[DennizGoren]

I forgot to mention the plane settings I used, so here they are. These settings provided me with the best results in terms of reflecting the surroundings. But come to think of it, I might scale things a bit down though to see how that influences performance.

[StijnOosterlinck]

Thanks @DennizGoren, I got the mirrors working.

I indeed noticed a huge impact on performance, I was wondering whether u know if the performance is any better when using a packaged version of carla instead of using it through UE? (so after running make package)

I get errors trying to run make package and I've not been able to solve them yet so I can not test this yet. Thanks!