PallHaraldsson
commented
4 years ago This issue kept me up last night, I had this nagging feeling I was giving bad advice, at https://discourse.julialang.org/t/best-way-to-optimize-code/37980/26 (I've followed up with the OP).
FYI, this is potentially a real issue for the code in that thread where my program, using your package, that used to be 3x faster goes to 80% slower, 2x and I guess slower still (without bounds? I just didn't have the patience to try larger values), with a one line change:
@time begin
nsim = 10^3;
omega = 1000; # was omega = 100, then faster, and I tried changing
[..]
67.861310 seconds (2.49 G allocations: 56.125 GiB, 3.85% gc time)
for omega=2000
276.214053 seconds (10.98 G allocations: 238.843 GiB, 5.59% gc time)
still more allocations for the original code:
139.043652 seconds (19.97 M allocations: 62.674 GiB, 2.09% gc time)I can still use your package (e.g. in that program for speedup, without the scalability issue.
chriselrod
A. First I want to start on a positive note.
Your RNG implementation is 2.5 times faster (probably just because the RNG is faster, despite vectorized).
While you also beat base on Float64 (by a smaller factor), it's interesting that still your Int RNG is slower than Float generation (you do not generate Float first, what I think Base does, so understandable for their).
While I remember, and int generation doesn't work with PCG (I don't care), should your tagline be changed (maybe you just started with PCG, and I maybe influenced you to add the other RNG): "Vectorized PCG uniform, normal, and exponential random samplers."
B. What you really want to show, is e.g. the possible 3.4 times gain, what you can ses in this microbenchmark, but I think maybe harmful:
Now, as you know, on discourse I've pointed to/used your package, and we could discuss this there in the open, but I want to make sure I'm not misunderstanding, before I give a warning about my program, 3x faster, mostly thanks to your package.
Someone else used this trick of precalculating the random numbers first, and I just improved that program keeping that idea. I've seen it before, maybe first on Debian's Benchmark game.
And it other do it, you are kind of forced to do so too in a microbenchmark to match other languages, since this works.
I just think this may be a very deceptive idea, and I think your vectorized package depends on it. At some point, for me around 2048 calculated random numbers it stops being faster, and gets slower with larger arrays, it seems/I guess around L1 cache amount of generated random numbers. But that's just the best case.
In all real-world programs you're going to do something with those values, and using the L1 cache for something else is valuable, and in those the right size might be much lower. So, is there a way to have your cake and eat it too?
I'm not sure what's a safe amount of generated random numbers. With just a few numbers generated you get almost all of the gain (maybe keep next for generated values in registers only?). But I think in programs where you use this idea, I'm not sure you can do away with a small buffer in memory.