Consider switching to XorShift for RNG

[Genbox]

XorShift is a simple alternative to the built-in Random class in C#, which provide better performance and randomness. I benchmarked the RNG implementations of Math.net as seen in the picture. The XorHome in the list is my own quick port of xoroshiro128+ from here: http://xoroshiro.di.unimi.it/xoroshiro128plus.c

As seen in the picture, the Math.Net XorShift is much faster than the built-in random.

[mdabros]

Hi @Genbox,

I'm not sure how much time the learners spent in the random generator during training. But if it is a significant part of the training time, it could be interesting to measure how much switching to XorShift would reduce it. I guess it will require some benchmarking to compare and see if a switch is worth the effort.

Best regards Mads

[Genbox]

Not much is spent on the RNG I'd wager. I have not profiled them, so I wouldn't know. However, I saw that the learners create a new Random object on every level in the trees, which means a lot of garbage is generated, which the GC then needs to pick up. A half-decent implementation of XorShift means faster random numbers, less garbage and better randomness.

There are definitely places that can benefit a lot more from a performance perspective, but the RNG seemed like a quick-win.