GordonBGood
commented
7 months ago Adding another comparison, F#/Fable, (an almost pure FP language other than it has optional mutability of all bindings, always mutable array contents, and uncontrolled IO side-effects) can transpile to Rust code from F# code, with the following code implementing the same benchmark as the OP:
let cLOOPS = 1000
let primesBench() =
let cmpsts = Array.zeroCreate 16384
let rec loopn n =
if n < cLOOPS then
let rec loopi i =
let s0 = (i + i) * (i + 3) + 3
if s0 < 131072 then
if cmpsts.[i >>> 3] &&& (1uy <<< (i &&& 7)) <> 0uy then loopi (i + 1) else
let bp = i + i + 3
let slmt = s0 + 8 * bp
let rec loops s =
if s < slmt then
let msk = 1uy <<< (s &&& 7)
let rec loopc c =
if c < 16384 then
cmpsts.[c] <- cmpsts.[c] ||| msk
loopc (c + bp)
loopc (s >>> 3); loops (s + bp)
loops s0; loopi (i + 1)
loopi 0; loopn (n + 1)
loopn 0
{0 .. 131071}
|> Seq.fold
(fun a i -> a - ((int cmpsts.[i >>> 3] >>> (i &&& 7)) &&& 1))
131073
[<EntryPoint>]
let main args =
printfn "PrimesBench in F#"
let start = System.DateTime.Now.Ticks
let answr = primesBench()
let elpsd = (System.DateTime.Now.Ticks - start) / 10000L
printfn "Found %d primes to 262146 for %d loops in %d milliseconds." answr cLOOPS elpsd
0On the same computer as in the OP, this has the following run times:
- Compiled to JavaScript (optimized): 197 milliseconds.
- Compiled to DotNet IR (Release mode): 213 milliseconds.
- Compiled to Rust (release/optimized): 169 milliseconds.
The above Rust results, which are a little faster than as run by JavaScript on Node.js or DotNet version 7.0 IR, are perhaps a better comparison to Roc as there is an automatic array bounds check per array access, which is one of the main reasons it is slower than the C version other than it also reloads the base array pointer on every inner culling loop (doesn't "lift" this outside the loop as it should). Using Rust's unsafe mutable pointers is not an option from Fable currently as it doesn't support the interop package that would allow that.
lukewilliamboswell
bhansconnect
Anton-4
Description
There have been some performance benchmarks written that show that Roc is "in there" as to running a simple QuickSort on various amounts of random data in spite of being purely functional using persistent data structures due to doing mutation in-place when such mutation cannot be detected "from the outside"; however, a random sorting algorithm doesn't really push low level speeds as much of the time is taken by missed branch predictions, as can be seen by the fastest compared language (C++) being less than twice as fast as the slowest compared language (JavaScript).
To properly evaluate this, a benchmark that is only dependent on raw low-level CPU computation speed is more appropriate, and for this I use a version of odds-only Sieve of Eratosthenes over a bit buffer no larger than the CPU L1 cache, which benchmark runs in C/C++ in about 1.5 CPU clock cycles per culling operation on a modern CPU. I made my best attempt at this in Roc and it is about 20 times slower at 30 CPU clock cycles per operation.
As Roc doesn't have something called contiguous arrays, I checked the implementation of Roc List's and that is what they actually are, as implied in some documentation. I will provide both my Roc code and C code for this benchmark so it can be seen that they both do the same thing. Some will immediately comment that I should profile the program to identify the "hot spot", but it is obvious that his program spends almost all of its time in the innermost simple culling loop. Memory management is also not an issue here as the benchmark program allocates just one List to run the benchmark over at least hundreds of milliseconds.
I am new to Roc, and it is possible that I am missing some techniques for optimization or compiler environment that must be provided for performance.
My Testing Environment
Fedora 39 Linux x86_64 with the prerequisites as per the installation instructions.
The nightly build of Roc as of 21 Jan 2024 decompressed with the File extract tool.
The benchmark example is put into the Examples folder inside the extracted folder, and the Roc compiler is run inside that folder with the following command: "../roc build --optimize prime-bench.roc" in a terminal.
The program is then run from a terminal as follows: "./prime-bench".
For C, the program is build using GCC as per the following command from a terminal: "gcc -O3 -o prime-bench prime-bench.c"
The C-built program is then run in the same way as the Roc-built one is.
The Benchmark Codes
Note that for both codes, the culling is run 1000 times for timing accuracy:
Roc (EDITED_TO_CLEAN_UP_CODE):
The equivalent C code:
The Results
For Roc: Found 23000 primes to 262146 in 1648 milliseconds.
For C: Found 23000 primes up to 262146 in 68.081000 milliseconds.
Currently, (Jan 21, 2024), Roc is over twenty times slower than the equivalent C code.
These results were as run on a AMD 7840HS CPU running at 5.1 GHz, and as there are about 200,000 culling operations per culling pass, there are about 200 million culling operations in the benchmark, so it can be calculated that it took about 1.73 CPU clock cycles per operation for the C version and about 42 CPU clock cycles per operation for the Roc version.
Expected Results
Roc benchmark times more in line with those of C.
Comments
This may be due to optimizations not yet enabled in Roc; for instance, if the call to the
List.updatefunction in the culling loop were not inlined, it would explain much of the slow-down for Roc. As well there may be overflow and/or bounds checks that are not being elided by the compiler as being redundant which could account for much or the remaining discrepancy.This benchmark may be useful to be added to the set of testing suites to check for performance progress and/or regressions.