Paged memory pool on CUDA

[stephen-hqxu]

Current

In STPHeightfieldGenerator, each time a new chunk needs to be computed, new device memory is allocated. However once the computation is done, memory is freed.

It may cause some memory allcation overhead if there are a lot of chunks being computed at the same time.

Suggestion

• The memory size is all fixed since our texture size is constant at runtime.
• Such that we can reuse those memory instead of freeing and reallocating them over and over again,

Approach

• Inspried by thread pool, we can have a paged memory pool.
• We have a number of allocated memory page in a queue.
• Pop one when requested.
• Insert back when done and no longer needed.
• If one thread wants a memory but there isn't more available, allocate more memory block, and insert into the pool when done using it.
• Clear up the memory pool regularly if there are way too many ununsed memory. Maybe everytime the a memory pointer is popped.

We need to have a function to measure how often and how much those allocated pages are used.

[stephen-hqxu]

Update

STPMemoryPool will be out in the next release, it supports customisable allocator and deallocator with user-defined passing arguments.

Heightfield generator has been implemented with the new memory pool, and I am pleased to say that even without legit benchmarking, the computational time has been cutdown by a lot, even noticeable by inspection.