Unbound chunk data usage

stephen-hqxu / superterrainplus

SuperTerrain+: A real-time procedural 3D infinite terrain engine with geographical features and photorealistic rendering.

MIT License

12 stars 1 forks source link

Unbound chunk data usage #7

Closed stephen-hqxu closed 3 years ago

stephen-hqxu commented 3 years ago

Current situation

New chunk data is always inserted to the hash table in STPChunkStorage. However chunks are kept until program exits.
This can cause problem such as out of memory if the program keeps running and new chunks are computed nonstopping.

Suggestion

Set a memory limit for the hash table.
Transfer the memory to secondary memory, a.k.a., hard drive if chunk hasn't been loaded for a long time.
Memory swap should be nonblocking. Thread should wait for chunk to be loaded from hard drive, instead it should keep rendering regardlessly (display blank chunk if not yet loaded like what we are doing right now).

stephen-hqxu commented 3 years ago

Update

Chunk serialisation has been implemented in the recent relase, chunk data can be manually exported and imported to and from the local storage.

Plan

Chunk serialisation is currently not automatically used by the program. We will be looking at some algorithm to serialise/deserialise data whenever it's necessary, for instance when STPChunkStorage hits a upper memory bound, and serialise the last 50% of least frequently referenced chunk.

stephen-hqxu commented 3 years ago

Abandon

Unfortunately we no longer find integrating serialisation into our engine useful as there are already libraries for similar purposes. To save our development time we decide to abandon this purposal.