search

gazebosim / gz-launch

Run and manage programs and plugins.
https://gazebosim.org
Apache License 2.0
10 stars 14 forks source link

:farmer: `CmdLine` test regressions on gz-launch5-win #202

Open Crola1702 opened 1 year ago

Crola1702 commented 1 year ago

Environment

Description

Steps to reproduce

  1. Run a job in ign_launch-win CI
  2. See test regressions fail.

Output

CmdLine.HelpVsCompletionFlags:

``` C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:136 Expected: (std::string::npos) != (helpOutput.find(flag)), actual: 18446744073709551615 vs 18446744073709551615 Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
' ```

CmdLine.Ls:

``` C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:70 Value of: output.find("CMakeFiles") != std::string::npos Actual: false Expected: true Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
' ```

CmdLine.EchoSelf:

``` C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:80 Expected equality of these values: filePath Which is: "C:\\Jenkins\\workspace\\ign_launch-ign-5-win\\ws\\gz-launch\\test\\config\\echo.ign" output Which is: "Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled\nC:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
'\n" With diff: @@ -1,1 +1,4 @@ -C:\\Jenkins\\workspace\\ign_launch-ign-5-win\\ws\\gz-launch\\test\\config\\echo.ign +Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled +C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) +\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' +\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
'\n Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
' ```

CmdLine.HelpSelf:

``` C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:88 Expected: (std::string::npos) != (output.find("Introspect Ignition launch")), actual: 18446744073709551615 vs 18446744073709551615 Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
' ```

CmdLine.EchoErb:

``` C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:97 Expected equality of these values: "erb1234" output Which is: "Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled\nC:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
'\n" With diff: @@ -1,1 +1,4 @@ -erb1234 +Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled +C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) +\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' +\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
'\n Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
' ```

CmdLine.EchoBadErb:

``` C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:106 Expected: (std::string::npos) != (output.find("is wrong for erb")), actual: 18446744073709551615 vs 18446744073709551615 Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL) from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute' from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `
' ```

They all have: ignition-tools-backward.dll not found in their output

Crola1702 commented 1 year ago

This happened today in https://build.osrfoundation.org/job/ign_launch-ign-5-win/38/console

azeey commented 1 year ago

It turns out, at least from what I can tell, none of our libraries, except ign-launch2 and ign-launch5 run UNIT_ign_TEST/UNIT_gz_TEST on Windows 😅. I'm going to investigate why, but if anyone knows, please chime in.

azeey commented 1 year ago

I was able to resolve the error regarding the invalid path:

C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
    from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
    from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

with https://github.com/azeey/gz-launch/commit/e6e7805810171f22ce267156f181c07c8396ef08, but now the UNIT_ign_TEST times out after 240s. Both the UNIT_Manager_TEST and now the UNIT_ign_TEST timeout while running dir or ls, which should return instantly. I'm also not sure if the UNIT_Manager_TEST ever passed. The original implementation (#120) indicates 6 failures, which is what we're seeing right now. So I propose we disable both tests on windows.

I'll try to apply https://github.com/azeey/gz-launch/commit/e6e7805810171f22ce267156f181c07c8396ef08 to other gz libraries and hopefully enable UNIT_ign_TEST/UNIT_gz_TEST.

Crola1702 commented 2 months ago

This issue is failing consistently in gz-launch5-win

Reference build: https://build.osrfoundation.org/job/gz_launch-ign-launch5-win/36/

Log output is different this time:

C:\J\workspace\gz_launch-ign-launch5-win\ws\gz-launch\src\cmd\ign_TEST.cc:136
Expected: (std::string::npos) != (helpOutput.find(flag)), actual: 18446744073709551615 vs 18446744073709551615
Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
    from C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
    from C:/J/workspace/gz_launch-ign-launch5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'