RoadInstantiatorSystem causes frame-rate spikes & Memory Allocations

[m3taphysics]

Explorer Alpha build versions:

Issue Description:

RoadInstantiatorSystem allocates memory especially when running around and moving. This can lead to GC spikes and spikes on instantiation.

STR:

1. Load genesis plaza
2. Load the profiler and filter by "RoadInstantiatorSystem"
3. Drop down the center and/or run around
4. As you drop down you should notice framerate spikes and larger memory allocactions

5. Expected behaviour:

Current behaviour:

Evidence:

Additional Notes:

[QThund]

Allocations occur when the object pool for a given road asset (there is a dictionary of asset<->pools) does not have enough cached objects and has to instantiate new ones. The only solutions seem to be either increasing the minimum cached objects or creating more than one object when there are not enough so we get the framerate spike less often. We could also try to release objects that are not in camera with the hope of reducing the chance of filling the pools, although this would surely produce popping; roads are unloaded as they are farther than a radius from the avatar, btw. Edit: Or another option could be to reduce the radius.

[QThund]

After studying the case, my conclussion is that the most reasonable solution is to cache N roads per pool and increase the amount of cached objects in a specific pool by X every time it gets full.

Pros:

• Memory allocation occurs less often.
• The first N instances are allocated while loading the scene so they can be used without allocating new memory since the beginning.
• After a short while, there is no memory allocation anymore.

Cons:

• More memory is reserved, even if it is never used.
• We have to assume that we will waste memory for sure. The problem with caching instances in advance is that not all pools store the same amount of road assets after a while: there may be one pool storing 3 instances whereas other is storing 150, so the first N-3 cached objects are using memory for nothing. N should be a value that is greater than the typical size of the pools and not too big that we waste too much memory.
• Every time a pool has to grow, the amount of memory that is reserved (X times the memory of the road asset) is bigger than adding just one instance. This may cause more noticiable hiccups. X cannot be too big.

Example of the contents of the road asset pool (N=30, X=20):

Name: Total cached (being used)
Corner_0: 30 (15)
Corner_A: 30 (20)
Corner_B: 30 (15)
Corner_C: 30 (21)
Corner_D: 30 (17)
Crossroads_00: 30 (2)
Crossroads_0: 30 (4)
Crossroads_A: 30 (3)
Crossroads_C: 30 (5)
DeadEnd_0: 30 (2)
EmptyFork_0: 30 (2)
EmptyFork_B: 30 (3)
EmptyFork_D: 30 (1)
Fork_0: 30 (13)
Fork_A: 30 (16)
Fork_B: 30 (20)
Fork_C: 30 (13)
Fork_D: 30 (16)
OpenCorner_0: 30 (2)
OpenCorner_B: 30 (2)
OpenCorner_C: 30 (1)
OpenFork_0: 30 (5)
OpenFork_A: 30 (8)
OpenFork_B: 30 (3)
OpenFork_C: 30 (1)
OpenRoad_0: 30 (30)
OpenRoad_A: 50 (31)
OpenRoad_B: 30 (28)
OpenRoad_C: 50 (31)
OpenRoad_D: 50 (31)
Road_0: 130 (112)
Road_A: 150 (137)
Road_B: 70 (66)
Road_C: 110 (97)
Road_D: 110 (98)
Road_E: 50 (50)
DeadEnd_A: 30 (1)
DeadEnd_C: 30 (3)
DeadEnd_E: 30 (2)
OpenCorner_A: 30 (2)
-49,-51: 30 (0)
-51,-49: 30 (1)
-70,-1: 30 (1)
0,-69: 30 (1)
0,12: 30 (1)
2,-51: 30 (0)
2,50: 30 (0)
13,0: 30 (1)

Additional thoughts: We could add more diversity to the most used road assets so there are more small pools and hence less memory allocations after loading the scene. For example, if Road_A requires 137 instances, there could be a Road_A1, Road_A2, Road_A3 and Road_A4 in the map, so pools could store 30, 30, 30, 30, 17.

As a real example, currently the extra memory we would need, using N=30 and X=20, would be 3Gb.

[QThund]

After a lot of testing and implementing different solutions (which include releasing road instances that were behind the camera), it seems that the most balanced solution so far is this:

• Road asset pools are pre-warmed when loading the world, caching 20 instances each (this value must respond to a balance between the memory usage and the chances that a pool gets filled). There is an exception to this: "long roads" (special assets formed by many roads) will not be cached in advance since that would require a lot of never-used memory, normally it is necessary to load only 1 or 2 of this kind. The system uses the tag "LongRoad" to identify them.
• Pools grow by tranches instead of one instance at a time. Current tranche size depends on the size of the pool, it equals a half of the total existing instances, in a range from 1 to 20. This reduces the chances of filling pools an hence FPS hiccups due to instantiation.
• The system will not filter out any road asset based on the orientation of the camera. That would produce popping due to marking roads as visible/not visible is an asynchronous process that is not fast enough to catch up the rotation of the camera. The amount of saved memory is not worth. Besides, enabling / disabling many gameobjects has also a cost.

The extra memory required for this solution is around 200 Mb. The amount of moments when FPS are affected by road instantiation is way lower (although could be more noticeable), and may occur during the first minute mostly.

A video just to illustrate popping: https://github.com/user-attachments/assets/b5cac978-04b5-4e2c-bd9f-f8cdd2372af0