Details on BLAS library used with R build

[she3o]

Hi, Thank you all for the informative benchmarks! The repo however does not clarify which BLAS implementation was used with the R build, and neither the Julia homepage clarifies that:

C and Fortran are compiled with gcc 7.3.1[...]. C, Fortran, Go, Julia, Lua, Python, and Octave use OpenBLAS v0.2.20 for matrix operations; Mathematica uses Intel® MKL.

There are several matrix libraries to build R with. the official manual for building R provides instructions to build R with OpenBLAS, Intel MKL, ATLAS, and the default BLAS library. in section A.3.1 of the manual:

The linear algebra routines in R can make use of enhanced BLAS [...] routines. However, these have to be explicitly requested at configure time: R provides an internal BLAS which is well-tested and will be adequate for most uses of R.

I tried to run perf.R on my machine with several build configurations for R (10 times each) and they show different results (up to x10 in one test):

R-BLAS-LAPACK

recursion_fibonacci,parse_integers,print_to_file,recursion_quicksort,userfunc_mandelbrot,iteration_pi_sum,matrix_statistics,matrix_multiply,
8.00000000,4.00000000,981.00000000,18.00000000,15.00000000,170.00000000,59.00000000,845.00000000
9.00000000,4.00000000,991.00000000,17.00000000,15.00000000,171.00000000,58.00000000,849.00000000
8.00000000,4.00000000,997.00000000,17.00000000,15.00000000,169.00000000,59.00000000,852.00000000
8.00000000,5.00000000,977.00000000,17.00000000,16.00000000,169.00000000,59.00000000,898.00000000
8.00000000,4.00000000,1004.00000000,17.00000000,15.00000000,169.00000000,59.00000000,863.00000000
9.00000000,4.00000000,985.00000000,17.00000000,15.00000000,173.00000000,59.00000000,862.00000000
9.00000000,5.00000000,972.00000000,17.00000000,15.00000000,179.00000000,59.00000000,852.00000000
8.00000000,4.00000000,960.00000000,17.00000000,16.00000000,170.00000000,58.00000000,881.00000000
8.00000000,4.00000000,961.00000000,17.00000000,15.00000000,168.00000000,58.00000000,838.00000000
8.00000000,4.00000000,995.00000000,17.00000000,15.00000000,169.00000000,59.00000000,843.00000000

R-MKL

8.00000000,4.00000000,1012.00000000,17.00000000,15.00000000,171.00000000,60.00000000,88.00000000
8.00000000,4.00000000,1000.00000000,17.00000000,15.00000000,177.00000000,60.00000000,88.00000000
8.00000000,4.00000000,992.00000000,17.00000000,15.00000000,171.00000000,60.00000000,88.00000000
8.00000000,4.00000000,993.00000000,17.00000000,16.00000000,172.00000000,60.00000000,87.00000000
8.00000000,4.00000000,982.00000000,17.00000000,15.00000000,171.00000000,60.00000000,88.00000000
8.00000000,4.00000000,1008.00000000,17.00000000,15.00000000,171.00000000,60.00000000,87.00000000
8.00000000,4.00000000,1017.00000000,18.00000000,14.00000000,181.00000000,58.00000000,86.00000000
8.00000000,4.00000000,1003.00000000,17.00000000,15.00000000,171.00000000,60.00000000,87.00000000
8.00000000,4.00000000,1016.00000000,17.00000000,15.00000000,228.00000000,59.00000000,87.00000000
8.00000000,4.00000000,980.00000000,17.00000000,15.00000000,179.00000000,60.00000000,89.00000000

R-OPENBLAS-LAPACK

8.00000000,4.00000000,1005.00000000,17.00000000,15.00000000,178.00000000,57.00000000,89.00000000
8.00000000,4.00000000,1003.00000000,18.00000000,15.00000000,198.00000000,57.00000000,88.00000000
8.00000000,4.00000000,1007.00000000,17.00000000,15.00000000,170.00000000,57.00000000,89.00000000
9.00000000,4.00000000,1014.00000000,17.00000000,15.00000000,177.00000000,57.00000000,89.00000000
8.00000000,4.00000000,1004.00000000,18.00000000,16.00000000,169.00000000,58.00000000,89.00000000
8.00000000,4.00000000,1023.00000000,17.00000000,14.00000000,172.00000000,57.00000000,90.00000000
8.00000000,4.00000000,993.00000000,17.00000000,15.00000000,183.00000000,57.00000000,89.00000000
8.00000000,4.00000000,1000.00000000,17.00000000,15.00000000,169.00000000,57.00000000,88.00000000
9.00000000,4.00000000,1007.00000000,17.00000000,16.00000000,171.00000000,58.00000000,90.00000000
9.00000000,4.00000000,1014.00000000,17.00000000,15.00000000,172.00000000,58.00000000,91.00000000

Intel also has benchmarks here showing differences between Intel MKL and OpenBLAS.

[StefanKarpinski]

We generally benchmark with whatever each project ships as official binaries. @johnfgibson ran the most recent set of benchmarks on a system he has access to and can give details about which R build he used.

Another point: BLAS benchmarks are particularly uninteresting as programming language comparisons because they primarily test how fast the BLAS library that is called is, not anything about the language. We really only included BLAS as a benchmark because people kept asking for a matrix multiply comparison.

[johnfgibson]

The R package used in those benchmarks is the standard x86-64 RPM R-3.5.0 package for openSUSE 15.0. On openSUSE 15.2 (which I'm running now) the standard R package uses its own libRbas.so. Probably 15.0 was the same.

The benchmarks use standard-install packages for all languages because that's what most people use. It's also a lot easier.

[acxz]

@she3o I believe your question has been answered. If you have another comments don't hesitate to comment.