speed slow , can I speed it up?

[ericyan71]

hello , I'm newer to xtensor, I really like it. yesterday, I made a test, found it is somewhat slow, below is consuming time of same code written in numpy, eigen and xtensor: numpy 1.5ms eigen 0.7ms xtensor 5.5ms

here is my code and test dada, I want to know if I made something wrong? thanks.

#include <iostream>
#include "xtensor/xarray.hpp"
#include "xtensor/xio.hpp"
#include "xtensor/xview.hpp"
#include "xtensor/xnpy.hpp"
#include <omp.h>

using namespace xt::placeholders;

void test()
{
    auto loc = xt::load_npy<float>("c:/temp/loc.npy");
    auto anchor = xt::load_npy<float>("c:/temp/anchor.npy");
    auto fg_score = xt::load_npy<float>("c:/temp/fg_score.npy");
    xt::xarray<float> result(loc.shape(), 0);

    int loop = 500;
    float ts = omp_get_wtime();
    for (int i = 0; i < loop; i++)
    {
        auto anc0 = xt::view(anchor, xt::all(), 0);
        auto anc1 = xt::view(anchor, xt::all(), 1);

        auto h = xt::view(anchor, xt::all(), 2) - anc0;
        auto w = xt::view(anchor, xt::all(), 3) - anc1;

        auto cy = anc0 + h / 2;
        auto cx = anc1 + w / 2;

        auto ty = h * xt::view(loc, xt::all(), 0) + cy;
        auto tx = w * xt::view(loc, xt::all(), 1) + cx;

        auto th = xt::exp(xt::view(loc, xt::all(), 2)) * h;
        auto tw = xt::exp(xt::view(loc, xt::all(), 3)) * w;

        xt::view(result, xt::all(), 0) = ty - th / 2;
        xt::view(result, xt::all(), 1) = tx - tw / 2;
        xt::view(result, xt::all(), 2) = ty + th / 2;
        xt::view(result, xt::all(), 3) = tx + tw / 2;
       }
    ts = omp_get_wtime() - ts;
    std::cout << ts/loop << std::endl;
}
int main() 
{
    test();
    return 0;
}

data0235425.zip

[SylvainCorlay]

• Did uou enable xsimd acceleration?
• You should get faster results by multiplying by 0.5f than dividing by 2.

[JohanMabille]

You can also avoid temporaries in the last assigns:

auto res0 = xt::view(result, xt::all(), 0);
auto res1 = xt::view(result, xt::all(), 1);
auto res2 = xt::view(result, xt::all(), 2);
auto res3 = xt::view(result, xt::all(), 3);

xt::noalias(res0) = ty - th / 2;
xt::noalias(res1) = tx - tw / 2;
xt::noalias(res2) = ty + th / 2;
xt::noalias(res3) = tx + tw / 2;

[wolfv]

If you have the Eigen code, we could use it to benchmark xtensor vs Eigen and make sure that we have comparable speeds.

[wolfv]

Also assigning the load_npy to a xtensor can help:

       #include <xtensor/xtensor.hpp>

    xt::xtensor<float, 2> loc = xt::load_npy<float>("c:/temp/loc.npy");
    xt::xtensor<float, 2> anchor = xt::load_npy<float>("c:/temp/anchor.npy");
    xt::xtensor<float, 2> fg_score = xt::load_npy<float>("c:/temp/fg_score.npy");

[ericyan71]

thanks, I followed above suggestion, and the consuming time drop to 3.8ms. I'm not sure the xsimd is used properly, I only add the xsimd include path to my project, and then set the compile flag XTENSOR_USE_XSIMD.

here is my Eigen code, the test data is the same as it used in xtensor:

#include <iostream>
#include <omp.h>
#include "Eigen/Core"
#include "xtensor/xio.hpp"
#include "xtensor/xnpy.hpp"

void load_data(Eigen::MatrixXf& mat_loc, Eigen::MatrixXf& mat_anchor)
{
    auto loc = xt::load_npy<float>("c:/temp/loc.npy");
    auto anchor = xt::load_npy<float>("c:/temp/anchor.npy");
    auto fg_score = xt::load_npy<float>("c:/temp/fg_score.npy");

    size_t d0 = loc.shape()[0];
    size_t d1 = loc.shape()[1];

    mat_loc.setZero(d0, d1);
    mat_anchor.setZero(d0, d1);

    for (int i = 0; i < d0; i++)
    {
        for (int j = 0; j < d1; j++)
        {
            mat_loc(i, j) = loc(i, j);
            mat_anchor(i, j) = anchor(i, j);
        }
    }
}

int main()
{
    Eigen::MatrixXf loc;
    Eigen::MatrixXf anchor;

    load_data(loc, anchor);

    Eigen::MatrixXf result(loc);
    result.setZero();

    float ts = omp_get_wtime();
    for (int i = 0; i < 40; i++)
    {
        auto top = anchor.col(0);
        auto left = anchor.col(1);

        auto h = anchor.col(2) - anchor.col(0);
        auto w = anchor.col(3) - anchor.col(1);

        auto cy = top + h / 2;
        auto cx = left + w / 2;

        auto ty = h.cwiseProduct(loc.col(0)) + cy;
        auto tx = w.cwiseProduct(loc.col(1)) + cx;

        auto th = h.cwiseProduct(loc.col(2).array().exp().matrix());
        auto tw = w.cwiseProduct(loc.col(3).array().exp().matrix());

        result.col(0) = ty - th / 2;
        result.col(1) = tx - tw / 2;
        result.col(2) = ty + th / 2;
        result.col(3) = tx + tw / 2;
    }
    ts = omp_get_wtime() - ts;
    std::cout << ts / 40 << std::endl;
    return 0;
}

[JohanMabille]

Also I think Eigen default layout is column major, could you try replacing the first lines of your test function with the following ones?

using tensor_type = xt::xtensor<float, 2, xt::layout_type::column_major>;
tensor_type loc = xt::load_npy<float>("c:/temp/loc.npy");
tensor_type anchor = xt::load_npy<float>("c:/temp/anchor.npy");
tensor_type fg_score = xt::load_npy<float>("c:/temp/fg_score.npy");
xt::xarray<float, xt::layout_type::column_major> result(loc.shape(), 0);

[SylvainCorlay]

Don't hesitate to share the latest xtensor version.

[ericyan71]

sorry I'm come back later and thank you very much for answering my questions with great patience. I tried again, I found when the variable 'result' is also change to tensor_type, and using column_major, the code get maximum speed of 0.9ms, that's great. if I use row_major, the speed is still 3.8ms.

I don't understand why column_major could be faster than row_major.

here is new code, thanks.

#include <iostream>
#include "xtensor/xarray.hpp"
#include "xtensor/xio.hpp"
#include "xtensor/xview.hpp"
#include "xtensor/xnpy.hpp"
#include "xtensor/xnoalias.hpp"
#include <omp.h>
#include <tuple>
#include <array>
#include "xtensor/xtensor.hpp"

using namespace xt::placeholders;

void test()
{
    using tensor_type = xt::xtensor<float, 2, xt::layout_type::column_major>;
    tensor_type loc = xt::load_npy<float>("c:/temp/loc.npy");
    tensor_type anchor = xt::load_npy<float>("c:/temp/anchor.npy");
    tensor_type fg_score = xt::load_npy<float>("c:/temp/fg_score.npy");
    //xt::xarray<float, xt::layout_type::column_major> result({ loc.shape()[0], loc.shape()[1] }, 0);
    tensor_type result(loc.shape(), 0);

    int loop = 500;
    float ts = omp_get_wtime();
    for (int i = 0; i < loop; i++)
    {
        auto anc0 = xt::view(anchor, xt::all(), 0);
        auto anc1 = xt::view(anchor, xt::all(), 1);

        auto h = xt::view(anchor, xt::all(), 2) - anc0;
        auto w = xt::view(anchor, xt::all(), 3) - anc1;

        auto cy = anc0 + h / 2;
        auto cx = anc1 + w / 2;

        auto ty = h * xt::view(loc, xt::all(), 0) + cy;
        auto tx = w * xt::view(loc, xt::all(), 1) + cx;

        auto th = xt::exp(xt::view(loc, xt::all(), 2)) * h;
        auto tw = xt::exp(xt::view(loc, xt::all(), 3)) * w;

        auto res0 = xt::view(result, xt::all(), 0);
        auto res1 = xt::view(result, xt::all(), 1);
        auto res2 = xt::view(result, xt::all(), 2);
        auto res3 = xt::view(result, xt::all(), 3);

        xt::noalias(res0) = ty - th / 2;
        xt::noalias(res1) = tx - tw / 2;
        xt::noalias(res2) = ty + th / 2;
        xt::noalias(res3) = tx + tw / 2;
    }
    ts = omp_get_wtime() - ts;
    std::cout << ts / loop << std::endl;
    std::cout << result << std::endl;
}

int main()
{
    test();
    return 0;
}

[SylvainCorlay]

You should be able to speed that up by replacing / 2 with * 0.5f.

The reason is that on scalar operations, the compiler would do the same with in both cases, but on arrays, it cannot optimize the division by a known quantity as a multiplication. Specifying f for the float prevents mixed arithmetics.

[SylvainCorlay]

The reason for column-major to be faster is that you are making operations on columns, which are memory-contiguous in column-major arrays, and allow for SIMD acceleration to kick in.

If you were manipulating rows, the row-major version would be faster.

[ericyan71]

OK, thanks ! I understand! and replacing /2 with *0.5f even speed it up, and the f should not be ignored. thanks again!

[JohanMabille]

You're welcome! Could you post the results of your final benchmarks?

[SylvainCorlay]

I am curious about your final benchmark!

[ericyan71]

ok, I add some code to show the result more clearly:

    ts = omp_get_wtime() - ts;
    std::cout << "time: " << ts / loop << "/s" << std::endl;
    std::cout << "result: " << std::endl;
    std::cout << result << std::endl;

and the result shows below: