`village_center.wbt` needs some optimizations

[ad-daniel]

Describe the Bug World village_center.wbt needs some optimizations. During the loading it goes through the processes:

1. Parsing Nodes
2. Downloading assets
3. Finalize nodes
4. Finalize nodes

During step (3), the memory consumption skyrockets to 12 GB (100%). Also, by reloading a few times, it doesn't always appear that step (2) gets consistently faster after caching. Some reloads appear to take as long as the first time.

On two occasions, after reloading a few times (and losing focus with the window while it was loading (not sure this is necessary for it to happen)), the Windows UI ended up crashing. Likely because of the memory usage.

I suspect a specific object might be the culprit as other big worlds don't suffer nearly as much.

This doesn't appear to be a regression however, the memory usage was already huge on master.

[lukicdarkoo]

it doesn't always appear that step (2) gets consistently faster after caching. Some reloads appear to take as long as the first time.

I assume it is because of many HTTP requests that are checking whether the files have changed online. This should never happen in our case, so I would disable this feature in the QT cache (if possible).

[omichel]

I would disable this feature in the QT cache (if possible).

This is indeed possible, but if a user changes the resource online, Webots will not use it and it will use the old one from its cache, which may not be desirable. Maybe we could add a preference item in the cache section called "cache policy" that would provide two options, like "prefer cache, prefer network" and would default to "prefer cache" to optimize performance, but could be turned to "prefer network" in case the online resources are changing.

[lukicdarkoo]

https://github.com/cyberbotics/webots/blob/2f78ab4c83ba4e71b578a4435b4cf45b245b6153/src/webots/nodes/utils/WbDownloader.cpp#L91-L92

I believe this should be replaced with:

request.setAttribute(QNetworkRequest::CacheLoadControlAttribute, QNetworkRequest::PreferCache);

Maybe we could add a preference item in the cache section called "cache policy" that would provide two options, like "prefer cache, prefer network" and would default to "prefer cache" to optimize performance, but could be turned to "prefer network" in case the online resources are changing.

Ideally yes, but for now they can use simple tricks to avoid the cache, like adding ?v=[version_number]: https://raw.githubusercontent.com/cyberbotics/webots/master/projects/default/worlds/textures/old_brick_wall.jpg?v=3

[BenjaminDeleze]

As found out in #5178, the church seems to be the problem (at least one of the proto that have issues) This seems to be due to the textures because changing all the geometries to simple boxes does not change anything.

We investigate a bit with @ygoumaz and so far:

• Removing the textures solves the problem (the proto does not take 500MB of RAM anymore)
• Decreasing the size of the textures (dividing them by 10) does not change anything.
• Changing the size of the boxes and scaling them have not effect as well.
• The church has 3 different appearances, if we apply the same appearance to the three shapes it decreases the RAM usage significantly.

We think it is an issue because the textures weight only a couple a MB and should be handle by the GPU.

[BenjaminDeleze]

We found the issue: the resolution. All the images of the church are 2048 x 2048. when loaded in Webots, qt uncompressed them because we need to have an array of pixels:

• To feed it to WREN
• To keep it for the pickColor function that is handled on the CPU side.

We use the QImage::Format_ARGB32 format. So the size of one image = 2048204832 = around 16MB We have 15 images for the Church.proto => 240MB of RAM!

We tested to reduce the resolution to 1024 x 1024 and the number matches: we dropped to 60MB of RAM.

So this is not a bug, but it is still annoying.

We see two possible solutions:

• Reduce the size of all images to 1024 or less. It should not be a problem for most of them, but it is a bit of shame that we cannot really support 2048.
• If possible transfer the pickColor on the GPU side such that we are not forced to keep the mImage after the parsing.