colgreen / sharpneat

SharpNEAT - Evolution of Neural Networks. A C# .NET Framework.
https://sharpneat.sourceforge.io/
Other
380 stars 97 forks source link

Make use of ArrayPool<T> to reduce memory allocs for neural net arrays. #48

Open colgreen opened 3 years ago

colgreen commented 3 years ago

When we decode a genome to a neural net we tend to allocate a few arrays. It would be good to try and use ArrayPool (probably, actually MemoryPool.Shared ?) where possible to minimise allocation and GC activity.

This work can be guided by the memory and GC tracking info from the performance profiler - we can use the efficacy sampler project to generate the workload for this.

Using arrays from a pool also means we avoid the memory clearing/zeroing overhead that occurs when getting a new array from the heap.

colgreen commented 3 years ago

Some more work towards this item has been done in this commit https://github.com/colgreen/sharpneat-refactor/commit/0f61c84a530abb73664ab152f83cbe58af2d5b40

colgreen commented 3 years ago

As a possible next step, we could rent the array used for connection weights in neural nets. This is a new array for acyclic nets, but is taken directly from the genome for cyclic nets. Hence, a wrapper type that is IDisposable can be used, with two versions, one which returns the array on disposal (for acyclic nets), and another that doesn't (for cyclic nets).

Further to that, we could also use ArrayPool for genome arrays instead of the current approach of trying to re-use arrays in offspring genomes where possible. That re-use means an array is not owned by any one genome, and thus we don't know when an array is ready to be returned to an ArrayPool (unless we reference count, which is probably going too far in terms of complexity of the codebase).