Suspicious Results

[idontgetoutmuch]

I would have expected PCG to be at least twice as fast as Mersenne and all the results seem to be identical for all the different RNGs. This makes me suspicious.

prng-bench $ ~/Library/Haskell/ghc-7.10.3/lib/prng-bench-0.1.0.0/bin/prng-bench
benchmarking Next/10000_Random
time                 122.8 μs   (121.6 μs .. 124.0 μs)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 122.6 μs   (121.1 μs .. 124.1 μs)
std dev              4.668 μs   (3.787 μs .. 5.908 μs)
variance introduced by outliers: 37% (moderately inflated)

benchmarking Next/10000_SplitMix
time                 122.9 μs   (121.3 μs .. 124.7 μs)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 122.7 μs   (121.5 μs .. 124.3 μs)
std dev              4.747 μs   (3.780 μs .. 6.969 μs)
variance introduced by outliers: 38% (moderately inflated)

benchmarking Next/10000_TFRandom
time                 120.8 μs   (119.5 μs .. 122.5 μs)
                     0.998 R²   (0.998 R² .. 0.999 R²)
mean                 121.5 μs   (120.3 μs .. 123.1 μs)
std dev              4.765 μs   (4.118 μs .. 5.649 μs)
variance introduced by outliers: 39% (moderately inflated)

benchmarking Next/10000_Mersenne
time                 122.5 μs   (121.2 μs .. 123.9 μs)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 123.2 μs   (122.2 μs .. 124.7 μs)
std dev              3.909 μs   (3.189 μs .. 5.051 μs)
variance introduced by outliers: 30% (moderately inflated)

benchmarking Next/10000_PCGRandom
time                 122.5 μs   (121.0 μs .. 123.9 μs)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 122.3 μs   (121.3 μs .. 123.7 μs)
std dev              4.147 μs   (3.377 μs .. 5.606 μs)
variance introduced by outliers: 32% (moderately inflated)

[nkartashov]

It does look suspicious, I will look into it. By the way, I used Ryan Yates' for testing PRNGs as well to show that SplitMix beats other generators. I uploaded the version from GSOC here.

[idontgetoutmuch]

I haven't yet tried SplitMix but here are results for Mersenne Twister, PCG and MWC. I am not surprised at Mersenne Twister being the slowest. I am surprised that MWC is so much faster than PCG. The test takes 10^7 Doubles and adds them.

SFMT
5000675.999132833
     886,211,160 bytes allocated in the heap
   1,164,321,584 bytes copied during GC
     336,503,088 bytes maximum residency (10 sample(s))
      12,768,976 bytes maximum slop
             546 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0      1372 colls,     0 par    0.596s   0.633s     0.0005s    0.0015s
  Gen  1        10 colls,     0 par    0.397s   0.561s     0.0561s    0.2850s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    0.573s  (  0.563s elapsed)
  GC      time    0.993s  (  1.194s elapsed)
  EXIT    time    0.004s  (  0.043s elapsed)
  Total   time    1.571s  (  1.799s elapsed)

  %GC     time      63.2%  (66.4% elapsed)

  Alloc rate    1,545,378,414 bytes per MUT second

  Productivity  36.8% of total user, 32.1% of total elapsed

PCG
5000338.525815555
     649,506,760 bytes allocated in the heap
   1,046,615,240 bytes copied during GC
     337,116,104 bytes maximum residency (10 sample(s))
      68,076,784 bytes maximum slop
             760 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0      1232 colls,     0 par    0.582s   0.658s     0.0005s    0.0020s
  Gen  1        10 colls,     0 par    0.403s   0.766s     0.0766s    0.5400s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    0.254s  (  0.278s elapsed)
  GC      time    0.984s  (  1.424s elapsed)
  EXIT    time    0.003s  (  0.069s elapsed)
  Total   time    1.243s  (  1.772s elapsed)

  %GC     time      79.2%  (80.4% elapsed)

  Alloc rate    2,552,460,514 bytes per MUT second

  Productivity  20.8% of total user, 14.6% of total elapsed

MWC
5001288.68368923
     880,123,088 bytes allocated in the heap
         294,568 bytes copied during GC
          44,312 bytes maximum residency (2 sample(s))
          50,208 bytes maximum slop
               1 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0      1679 colls,     0 par    0.004s   0.005s     0.0000s    0.0006s
  Gen  1         2 colls,     0 par    0.000s   0.000s     0.0002s    0.0003s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    0.211s  (  0.217s elapsed)
  GC      time    0.004s  (  0.005s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    0.216s  (  0.222s elapsed)

  %GC     time       1.8%  (2.3% elapsed)

  Alloc rate    4,171,298,315 bytes per MUT second

  Productivity  98.2% of total user, 95.4% of total elapsed

[nkartashov]

Which one of the PCG from Ryan's test (if any) did you run?