xtensor-stack / xtensor

C++ tensors with broadcasting and lazy computing
BSD 3-Clause "New" or "Revised" License
3.36k stars 398 forks source link

arm64 build timing out on test_xblockwise_reducer.cpp #2770

Open drew-parsons opened 10 months ago

drew-parsons commented 10 months ago

The debian build is having a timeout problem building CI tests for xtensor 0.24.7 on arm64. I don't think it's an actual bug, but I'd like to hear your recommendations for handling it.

The failing test log is https://ci.debian.net/data/autopkgtest/testing/arm64/x/xtensor/41963554/log.gz

The relevant excerpt is

 48s [  8%] Building CXX object CMakeFiles/test_xtensor_lib.dir/test_xblockwise_reducer.cpp.o
 48s /usr/bin/g++   -DXSIMD_ENABLE_XTL_COMPLEX=1 -march=native -std=c++14 -Wunused-parameter -Wextra -Wreorder -Wconversion -Wno-sign-conversion  -Wold-style-cast -Wunused-variable -ftemplate-backtrace-limit=0 -O3 -DNDEBUG -MD -MT CMakeFiles/test_xtensor_lib.dir/test_xblockwise_reducer.cpp.o -MF CMakeFiles/test_xtensor_lib.dir/test_xblockwise_reducer.cpp.o.d -o CMakeFiles/test_xtensor_lib.dir/test_xblockwise_reducer.cpp.o -c /tmp/autopkgtest-lxc.bxn_6p7d/downtmp/autopkgtest_tmp/test_xblockwise_reducer.cpp
 51s [  8%] Building CXX object CMakeFiles/test_xtensor_lib.dir/test_xbroadcast.cpp.o
 51s /usr/bin/g++   -DXSIMD_ENABLE_XTL_COMPLEX=1 -march=native -std=c++14 -Wunused-parameter -Wextra -Wreorder -Wconversion -Wno-sign-conversion  -Wold-style-cast -Wunused-variable -ftemplate-backtrace-limit=0 -O3 -DNDEBUG -MD -MT CMakeFiles/test_xtensor_lib.dir/test_xbroadcast.cpp.o -MF CMakeFiles/test_xtensor_lib.dir/test_xbroadcast.cpp.o.d -o CMakeFiles/test_xtensor_lib.dir/test_xbroadcast.cpp.o -c /tmp/autopkgtest-lxc.bxn_6p7d/downtmp/autopkgtest_tmp/test_xbroadcast.cpp
 53s [  8%] Building CXX object CMakeFiles/test_xtensor_lib.dir/test_xbuilder.cpp.o
 53s /usr/bin/g++   -DXSIMD_ENABLE_XTL_COMPLEX=1 -march=native -std=c++14 -Wunused-parameter -Wextra -Wreorder -Wconversion -Wno-sign-conversion  -Wold-style-cast -Wunused-variable -ftemplate-backtrace-limit=0 -O3 -DNDEBUG -MD -MT CMakeFiles/test_xtensor_lib.dir/test_xbuilder.cpp.o -MF CMakeFiles/test_xtensor_lib.dir/test_xbuilder.cpp.o.d -o CMakeFiles/test_xtensor_lib.dir/test_xbuilder.cpp.o -c /tmp/autopkgtest-lxc.bxn_6p7d/downtmp/autopkgtest_tmp/test_xbuilder.cpp
...
 92s [ 26%] Building CXX object CMakeFiles/test_xtensor_lib.dir/test_xoperation.cpp.o
 92s /usr/bin/g++   -DXSIMD_ENABLE_XTL_COMPLEX=1 -march=native -std=c++14 -Wunused-parameter -Wextra -Wreorder -Wconversion -Wno-sign-conversion  -Wold-style-cast -Wunused-variable -ftemplate-backtrace-limit=0 -O3 -DNDEBUG -MD -MT CMakeFiles/test_xtensor_lib.dir/test_xoperation.cpp.o -MF CMakeFiles/test_xtensor_lib.dir/test_xoperation.cpp.o.d -o CMakeFiles/test_xtensor_lib.dir/test_xoperation.cpp.o -c /tmp/autopkgtest-lxc.bxn_6p7d/downtmp/autopkgtest_tmp/test_xoperation.cpp
 95s [ 26%] Building CXX object CMakeFiles/test_xtensor_lib.dir/test_xoptional_assembly.cpp.o
 95s /usr/bin/g++   -DXSIMD_ENABLE_XTL_COMPLEX=1 -march=native -std=c++14 -Wunused-parameter -Wextra -Wreorder -Wconversion -Wno-sign-conversion  -Wold-style-cast -Wunused-variable -ftemplate-backtrace-limit=0 -O3 -DNDEBUG -MD -MT CMakeFiles/test_xtensor_lib.dir/test_xoptional_assembly.cpp.o -MF CMakeFiles/test_xtensor_lib.dir/test_xoptional_assembly.cpp.o.d -o CMakeFiles/test_xtensor_lib.dir/test_xoptional_assembly.cpp.o -c /tmp/autopkgtest-lxc.bxn_6p7d/downtmp/autopkgtest_tmp/test_xoptional_assembly.cpp
748s g++: fatal error: Killed signal terminated program cc1plus
748s compilation terminated.
748s make[3]: *** [CMakeFiles/test_xtensor_lib.dir/build.make:132: CMakeFiles/test_xtensor_lib.dir/test_xblockwise_reducer.cpp.o] Error 1

The integer at the front of the line is timing in seconds. So compilation of test_xblockwise_reducer.cpp.o starts at 48s and is killas at 748s. Evidently the test environment has a gcc timeout of 700s. I note also the previous step before killing was launching compilation of test_xoperation.cpp.o at 92s, so many of the compilation processes are taking a long time on arm64.

For comparison, the same test on amd64 starts at 26s and reaches linking at 87s.

Are there any known issues with compilation time on arm64? What would you recommend for managing this problem? Some options that come to mind,

I'll be happy to test your recommendations.

drew-parsons commented 10 months ago

It's possibly a problem with the debian arm64 server. A similar problem is happening in another package. I'll keep monitoring the situation.