search

godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
89.11k stars 20.2k forks source link

MemoryPool size hardcoded? #37154

Open Anyeos opened 4 years ago

Anyeos commented 4 years ago

Godot version: 3.2.1rc1-custom

OS/device including version: Ubuntu 19.10

Issue description: Why MemoryPool amount is hardcoded?

Just on the file core/pool_vector.h there are the declaration: static void setup(uint32_t p_max_allocs = (1 << 16));

I am asking if there are a reason to do that. I need more memory for a module that create voxels and I will change it for myself. But I see that in Godot 4.0 the memory pool is rewrited. So I hope it can allocate by demand. Am I right about this? And will the 3.2.x series remain hardcoded? Or are there some plans to make that value a project setting?

aaronfranke commented 4 years ago

And will the 3.2.x series remain hardcoded?

Yes. It would be extremely unlikely for something this low-level to be changed in a stable branch.

Anyeos commented 4 years ago

Maybe can be implemented some "advanced" and "only for experienced users" tab under the settings window And will print a warning if someone wants to change something there. It is just an idea.

Calinou commented 4 years ago

@Anyeos We don't have a way to signal settings as "advanced" yet, but this may be added in 4.0.

akien-mga commented 4 years ago

I don't think it could be a setting anyway, as PoolVector is too low level and you can't access Project Settings before having it initialized. This would have to be set a compile time.

You're also the first user to hit a limit on the MemoryPool allocs AFAIR, and the problem is fixed in 4.0 as there's no MemoryPool anymore, so I suggest you do a custom build with an increased value (or since it's a custom module, maybe you can work it around with e.g. std::vector?).

Anyeos commented 4 years ago

I don't think it could be a setting anyway, as PoolVector is too low level and you can't access Project Settings before having it initialized. This would have to be set a compile time.

You're also the first user to hit a limit on the MemoryPool allocs AFAIR, and the problem is fixed in 4.0 as there's no MemoryPool anymore, so I suggest you do a custom build with an increased value (or since it's a custom module, maybe you can work it around with e.g. std::vector?).

I agree. But I think there will be more users hitting memory limit if they use 3D under a platform with more power than a movile device. Anyway always this kind of problems I try to solve compiling a custom build.

I hope Godot 4 can allocate memory dynamically instead statically for that purpose at least.

The module is https://github.com/Zylann/godot_voxel and it is not mine. But I just wanted to deploy a big voxel terrain and that is how I hit the memory limit so easily.

Demindiro commented 3 years ago

I just encountered this issue while running a test scene. In my particular case, I'm calling Mesh.surface_get_arrays() on a few meshes very often and I'm storing the results in an array.

Since in my case a lot of the arrays are effectively the same, I worked around this issue by caching all Pool*Arrays and if a duplicate is found in the cache, the cached version is returned and the other array is discarded.

Example code for a caching singleton

```gdscript extends Node var _cache := {} func dedup_immutable(object): if object in _cache: return _cache[object] else: _cache[object] = object return object ```

Demindiro commented 3 years ago

Related: #8279

greenlion commented 3 years ago

I had the same problem with large voxel maps. I think with more folks doing memory intensive things that this should probably be fixed. To change the code took all of about 30 minutes and a little testing. I can submit a PR if you want.

Calinou commented 3 years ago

I can submit a PR if you want.

Please do, but keep in mind core structures have changed significantly in the master branch. Therefore, you'll probably have to open a pull request against the 3.x branch and make sure it doesn't break compatibility with existing projects.

greenlion commented 3 years ago

Since 4.x doesn't contain the code anymore that would be pretty tough. :D

On Tue, Jun 15, 2021 at 6:32 PM Hugo Locurcio @.***> wrote:

I can submit a PR if you want.

Please do, but keep in mind pull requests should be opened against the master branch before being opened against the 3.x branch for backporting 🙂

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/37154#issuecomment-861878405, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADRTNQZ2BJS6GWBUYOXEYDTS7IH7ANCNFSM4LOYAB3Q .