Open ansar-sa opened 3 years ago
I think there are two differences with eigen:
dot
operation, there is some overhead to check if you're broadcastingIt could be interesting to make a 2D dot function that works faster on small matrices / matrix-vector combination. We could even use Eigen as a "backend" for that :)
It could be interesting to make a 2D dot function that works faster on small matrices / matrix-vector combination. We could even use Eigen as a "backend" for that :)
This is an interesting idea. First, two thoughts:
add_definitions(-DHAVE_CBLAS=1 -DWITH_OPENBLAS=1 -DXTENSOR_ENABLE_XSIMD=1)
set(LINK_LIBRARIES ${CONAN_LIBS} ${BLAS_LIBRARIES} ${LAPACK_LIBRARIES} xtensor xtensor::optimize xtensor::use_xsimd)
set(RELEASE_FLAGS "-Ofast -DNDEBUG -march=native ${WARNING_FLAGS}")
MKL
or do you think that will be just as slow for small matrices? In case both options are a no go, yes I would love to see xtensor
handle small matrix / matrix, matrix / vec dot products fast either via handrolled simd code or via eigen backend.
As I mentioned before, the dot product is more general than Eigen's matrix / matrix-vector multiplication as it performs broadcasting. Without broadcasting checks you'd probably already be 2x faster for small matrices.
You could try using xtensor
as a class. It has a compile time dimension (just as MatrixXd is always 2-dimensionla) you can use xtensor<double, 2>
.
MKL won't help much I am afraid :)
calling into BLAS functions has some intrinsic overhead (since even a function call vs. inlined cost has some cost attached to it).
Using xtensor class does not help either. As you say, the call into BLAS and broadcast checks is probably dominating the time with small matrices. Do you have some example code on how I can use xsimd to multiply a small matrix with a vector please?
Actually the problem is in other kinds of operations as well. Even simple array ops such as element wise array sums for xtensor based code is very slow compared to eigen when it comes to small arrays. Can we please look what bounds / broadcast checks are the culprit. Benchmark results:
Running ./array_ops_performance_test
Run on (6 X 2900 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x6)
L1 Instruction 32 KiB (x6)
L2 Unified 256 KiB (x6)
L3 Unified 12288 KiB (x6)
Load Average: 0.02, 0.64, 0.73
----------------------------------------------------------------
Benchmark Time CPU Iterations
----------------------------------------------------------------
eigen_array_sum/2 0.857 ns 0.854 ns 840056658
eigen_array_sum/4 3.61 ns 3.61 ns 197734174
eigen_array_sum/8 7.89 ns 7.88 ns 82837277
eigen_array_sum/16 36.7 ns 36.7 ns 19514002
eigen_array_sum/32 153 ns 153 ns 4527008
eigen_array_sum/64 1117 ns 1115 ns 627337
eigen_array_sum/128 6368 ns 6357 ns 102095
eigen_array_sum/256 32093 ns 32039 ns 24341
eigen_array_sum/512 155964 ns 155738 ns 4763
xtensor_array_sum/2 133 ns 132 ns 5450911
xtensor_array_sum/4 142 ns 141 ns 4697384
xtensor_array_sum/8 150 ns 150 ns 4680743
xtensor_array_sum/16 249 ns 249 ns 2787851
xtensor_array_sum/32 544 ns 542 ns 1326147
xtensor_array_sum/64 1954 ns 1950 ns 343751
xtensor_array_sum/128 8772 ns 8759 ns 74394
xtensor_array_sum/256 35809 ns 35724 ns 19592
xtensor_array_sum/512 113890 ns 113503 ns 6039
#include <benchmark/benchmark.h>
#include <Eigen/Dense>
#include <xtensor/xarray.hpp>
using namespace Eigen;
static void eigen_array_sum(benchmark::State &state) {
int size = state.range(0);
MatrixXd a = MatrixXd::Random(size, size);
MatrixXd b = MatrixXd::Random(size, size);
MatrixXd r = MatrixXd::Random(size, size);
for (auto _ : state) {
r = a + b;
}
}
BENCHMARK(eigen_array_sum)->RangeMultiplier(2)->Range(2, 8 << 6);
static void xtensor_array_sum(benchmark::State &state) {
uint64_t size = state.range(0);
xt::xarray<double>::shape_type shape = {size, size};
xt::xarray<double> a(shape);
xt::xarray<double> b(shape);
xt::xarray<double> r(shape);
for (auto _ : state) {
r = a + b;
}
}
BENCHMARK(xtensor_array_sum)->RangeMultiplier(2)->Range(2, 8 << 6);
BENCHMARK_MAIN();
comparing MatrixXd
and xt::xarray<double>
is never a fair comparison. Quite different containers (since xarray is dynamically ndimensional, MatrixXd is statically 2d).
I'll update benchmark results using xtensor<double, 2>
-------------------------------------------------------------------------
Benchmark Time CPU Iterations
-------------------------------------------------------------------------
eigen_array_sum/2 1.29 ns 1.28 ns 457478259
eigen_array_sum/4 3.31 ns 3.29 ns 208422955
eigen_array_sum/8 6.83 ns 6.80 ns 96818924
eigen_array_sum/16 32.8 ns 32.6 ns 20155042
eigen_array_sum/32 138 ns 138 ns 4957244
eigen_array_sum/64 1009 ns 1008 ns 652397
eigen_array_sum/128 6065 ns 6060 ns 112739
eigen_array_sum/256 27283 ns 27275 ns 25480
eigen_array_sum/512 120261 ns 120255 ns 5317
xtensor_array_sum/2 34.4 ns 34.4 ns 20148667
xtensor_array_sum/4 37.2 ns 37.2 ns 18514938
xtensor_array_sum/8 50.0 ns 50.0 ns 12796309
xtensor_array_sum/16 139 ns 139 ns 4996273
xtensor_array_sum/32 392 ns 392 ns 1787989
xtensor_array_sum/64 1740 ns 1738 ns 406974
xtensor_array_sum/128 7859 ns 7858 ns 81830
xtensor_array_sum/256 32434 ns 32432 ns 21366
xtensor_array_sum/512 104275 ns 104264 ns 6603
static void eigen_array_sum(benchmark::State &state) {
int size = state.range(0);
MatrixXd a = MatrixXd::Random(size, size);
MatrixXd b = MatrixXd::Random(size, size);
MatrixXd r = MatrixXd::Random(size, size);
for (auto _ : state) {
r = a + b;
}
}
BENCHMARK(eigen_array_sum)->RangeMultiplier(2)->Range(2, 8 << 6);
static void xtensor_array_sum(benchmark::State &state) {
uint64_t size = state.range(0);
xt::xtensor<double, 2>::shape_type shape = {size, size};
xt::xtensor<double, 2> a(shape);
xt::xtensor<double, 2> b(shape);
xt::xtensor<double, 2> r(shape);
for (auto _ : state) {
r = a + b;
}
}
BENCHMARK(xtensor_array_sum)->RangeMultiplier(2)->Range(2, 8 << 6);
you can shave off some more ns by doing xt::noalias(r) = a + b;
There might be some similar trick for eigen.
Benchmark results:
Benchmark code:
Any thoughts why small arrays for xtensor incur so much overhead? what should I do if most of my arrays are small (but not fixed size).