flexible-collision-library / fcl

Flexible Collision Library
https://flexible-collision-library.github.io/
Other
1.43k stars 417 forks source link

test_fcl_simple timeouts #599

Open SeanCurtis-TRI opened 1 year ago

SeanCurtis-TRI commented 1 year ago

In windows CI (on appveyor), the test_fcl_simple test has been known to timeout (see reproduced log below).

Generally, the test has issues; it is not a proper test. Here is a list of defects immediately obvious upon initial read (the list is not comprehensive):

  1. It performs test on Eigen types (e.g., Vectorn<S, N> is just an alias for Eigen::Matrix<S, N, 1>, but we test all the matrix operations).
  2. Many of those tests consist simply of dumping values to std::out.
  3. In addition, there are examples of the test running on uninitialized values.

So, while it's not immediately obvious what would be causing the timeouts, it seems safe to say that some clean up is due. The clean up may assist in time out problem (and certainly couldn't hurt).

[00:16:51]      1>Project "C:\projects\fcl\build\fcl.sln" (1) is building "C:\projects\fcl\build\test\test_fcl_simple.vcxproj.metaproj" (38) on node 1 (default targets).
[00:16:51]     38>Project "C:\projects\fcl\build\test\test_fcl_simple.vcxproj.metaproj" (38) is building "C:\projects\fcl\build\test\test_fcl_simple.vcxproj" (77) on node 2 (default targets).
[00:16:51]     77>PrepareForBuild:
[00:16:51]          Creating directory "test_fcl_simple.dir\Release\".
[00:16:51]          Creating directory "test_fcl_simple.dir\Release\test_fcl_simple.tlog\".
[00:16:51]        InitializeBuildStatus:
[00:16:51]          Creating "test_fcl_simple.dir\Release\test_fcl_simple.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
[00:16:51]        CustomBuild:
[00:16:51]          Building Custom Rule C:/projects/fcl/test/CMakeLists.txt
[00:16:51]        VcpkgTripletSelection:
[00:16:51]          Using triplet "x64-windows" from "C:\Tools\vcpkg\installed\x64-windows\"
[00:16:51]        ClCompile:
[00:16:51]          C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\CL.exe /c /IC:\projects\fcl\test /IC:\projects\fcl\test\gtest /IC:\projects\fcl\test\gtest\include /IC:\projects\fcl\build\include /IC:\projects\fcl\include /IC:\projects\fcl\test\. /IC:\projects\fcl\build\test /I"C:\Program Files\libccd\include" /I"C:\Program Files\Eigen\include\eigen3" /I"C:\projects\fcl\build\_deps\googletest-src\googletest\include" /I"C:\projects\fcl\build\_deps\googletest-src\googletest" /I"C:\Tools\vcpkg\installed\x64-windows\include" /nologo /W1 /WX- /MP /O2 /Ob2 /D WIN32 /D _WINDOWS /D NDEBUG /D "CMAKE_INTDIR=\"Release\"" /D _MBCS /Gm- /EHsc /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /Fo"test_fcl_simple.dir\Release\\" /Fd"test_fcl_simple.dir\Release\vc140.pdb" /Gd /TP /errorReport:queue  /bigobj -std:c++14 C:\projects\fcl\test\test_fcl_simple.cpp
[00:17:03]          test_fcl_simple.cpp
[00:17:53]        Link:
[00:17:53]          C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\link.exe /ERRORREPORT:QUEUE /OUT:"C:\projects\fcl\build\test\Release\test_fcl_simple.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:C:/projects/fcl/build/test /LIBPATH:C:/projects/fcl/build/test/Release /LIBPATH:"C:\Tools\vcpkg\installed\x64-windows\lib" /LIBPATH:"C:\Tools\vcpkg\installed\x64-windows\lib\manual-link" ..\lib\Release\fcl.lib ..\lib\Release\test_fcl_utility.lib ..\lib\Release\gtest_main.lib ..\lib\Release\fcl.lib "C:\Program Files\libccd\lib\ccd.lib" ..\lib\Release\gtest.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:"C:/projects/fcl/build/test/Release/test_fcl_simple.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/projects/fcl/build/test/Release/test_fcl_simple.lib" /MACHINE:X64  /machine:x64 test_fcl_simple.dir\Release\test_fcl_simple.obj
[00:18:38]          test_fcl_simple.vcxproj -> C:\projects\fcl\build\test\Release\test_fcl_simple.exe
[00:18:38]        AppLocalFromInstalled:
[00:18:38]          C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -noprofile -File "C:\Tools\vcpkg\scripts\buildsystems\msbuild\applocal.ps1" "C:\projects\fcl\build\test\Release\test_fcl_simple.exe" "C:\Tools\vcpkg\installed\x64-windows\bin" "test_fcl_simple.dir\Release\test_fcl_simple.tlog\test_fcl_simple.write.1u.tlog" "test_fcl_simple.dir\Release\vcpkg.applocal.log"
[00:19:19]     74>Link:
[00:19:19]          C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\link.exe /ERRORREPORT:QUEUE /OUT:"C:\projects\fcl\build\test\Release\test_fcl_sphere_cylinder.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:C:/projects/fcl/build/test /LIBPATH:C:/projects/fcl/build/test/Release /LIBPATH:"C:\Tools\vcpkg\installed\x64-windows\lib" /LIBPATH:"C:\Tools\vcpkg\installed\x64-windows\lib\manual-link" ..\lib\Release\fcl.lib ..\lib\Release\test_fcl_utility.lib ..\lib\Release\gtest_main.lib ..\lib\Release\fcl.lib "C:\Program Files\libccd\lib\ccd.lib" ..\lib\Release\gtest.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:"C:/projects/fcl/build/test/Release/test_fcl_sphere_cylinder.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/projects/fcl/build/test/Release/test_fcl_sphere_cylinder.lib" /MACHINE:X64  /machine:x64 test_fcl_sphere_cylinder.dir\Release\test_fcl_sphere_cylinder.obj
[00:19:30]          test_fcl_sphere_cylinder.vcxproj -> C:\projects\fcl\build\test\Release\test_fcl_sphere_cylinder.exe
[00:19:30]        AppLocalFromInstalled:
[00:19:30]          C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -noprofile -File "C:\Tools\vcpkg\scripts\buildsystems\msbuild\applocal.ps1" "C:\projects\fcl\build\test\Release\test_fcl_sphere_cylinder.exe" "C:\Tools\vcpkg\installed\x64-windows\bin" "test_fcl_sphere_cylinder.dir\Release\test_fcl.02D98944.tlog\test_fcl_sphere_cylinder.write.1u.tlog" "test_fcl_sphere_cylinder.dir\Release\vcpkg.applocal.log"
[00:19:38]     77>PostBuildEvent:
[00:19:39]          setlocal
[00:19:39]          "C:\Program Files (x86)\CMake\bin\cmake.exe" -D TEST_TARGET=test_fcl_simple -D TEST_EXECUTABLE=C:/projects/fcl/build/test/Release/test_fcl_simple.exe -D TEST_EXECUTOR= -D TEST_WORKING_DIR=C:/projects/fcl/build/test -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=test_fcl_simple_TESTS -D CTEST_FILE=C:/projects/fcl/build/test/test_fcl_simple[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -P "C:/Program Files (x86)/CMake/share/cmake-3.16/Modules/GoogleTestAddTests.cmake"
[00:19:39]          if %errorlevel% neq 0 goto :cmEnd
[00:19:39]          :cmEnd
[00:19:39]          endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
[00:19:39]          :cmErrorLevel
[00:19:39]          exit /b %1
[00:19:39]          :cmDone
[00:19:39]          if %errorlevel% neq 0 goto :VCEnd
[00:19:39]          :VCEnd
[00:19:53]          CMake Error at C:/Program Files (x86)/CMake/share/cmake-3.16/Modules/GoogleTestAddTests.cmake:40 (message):
[00:19:53]            Error running test executable.
[00:19:54]          
[00:19:54]              Path: 'C:/projects/fcl/build/test/Release/test_fcl_simple.exe'
[00:19:54]              Result: Process terminated due to timeout
[00:19:54]              Output:
[00:19:54]                FCL_SIMPLE.
[00:19:54]                  Vec_nf_test
[00:19:54]                  projection_test_line
[00:19:54]                  projection_test_triangle
[00:19:54]                  projection_test_tetrahedron
[00:19:54]                
[00:19:54]          
SeanCurtis-TRI commented 1 year ago

The timeout documented above occurred while working on #598. Simply retriggering the single failed test was enough to get it to pass.

However, the post merge CI check had a corresponding failure. This time in test_fcl_sphere_capsule:

75>PostBuildEvent:
         setlocal
         "C:\Program Files (x86)\CMake\bin\cmake.exe" -D TEST_TARGET=test_fcl_sphere_capsule -D TEST_EXECUTABLE=C:/projects/fcl/build/test/Release/test_fcl_sphere_capsule.exe -D TEST_EXECUTOR= -D TEST_WORKING_DIR=C:/projects/fcl/build/test -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=test_fcl_sphere_capsule_TESTS -D CTEST_FILE=C:/projects/fcl/build/test/test_fcl_sphere_capsule[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -P "C:/Program Files (x86)/CMake/share/cmake-3.16/Modules/GoogleTestAddTests.cmake"
         if %errorlevel% neq 0 goto :cmEnd
         :cmEnd
         endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
         :cmErrorLevel
         exit /b %1
         :cmDone
         if %errorlevel% neq 0 goto :VCEnd
         :VCEnd
         CMake Error at C:/Program Files (x86)/CMake/share/cmake-3.16/Modules/GoogleTestAddTests.cmake:40 (message):
           Error running test executable.

             Path: 'C:/projects/fcl/build/test/Release/test_fcl_sphere_capsule.exe'
             Result: Process terminated due to timeout
             Output:

The problem may be the antiquated compiler platform. Possibly need to update the build target in appveyor.