search

longturn / freeciv21

Develop your civilization from humble roots to a global empire
GNU General Public License v3.0
219 stars 42 forks source link

Unit Tests fail on Windows Msys2 #2122

Open jwrober opened 9 months ago

jwrober commented 9 months ago

Describe the bug Unit Tests fail on Windows Msys2

To Reproduce Steps to reproduce the behavior:

  1. Configure and build fc21 on Msys2
  2. Starts the test cmake --build build -target test
  3. See error

Expected behavior We should be able to run tests on windows.

Some logs...

The server can be invoked from the Msys2 terminal

James@DESKTOP-2EQQ3A9 MINGW64 ~/Freeciv21
$ build/Debug/freeciv21-server.exe
Encodings: Data=UTF-8, Local=en_US, Internal=UTF-8
This is the server for Freeciv21 version 3.1.7911089.4-alpha
You can learn a lot about Freeciv21 at https://longturn.readthedocs.io/en/latest/
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/sernet.cpp:394) - Server attempting to listen on :5556
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/ruleset.cpp:8661) - Loading rulesets.
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/srv_main.cpp:2493) - AI*1 has been added as Easy level AI-controlled player (classic).
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/srv_main.cpp:2493) - AI*2 has been added as Easy level AI-controlled player (classic).
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/srv_main.cpp:2493) - AI*3 has been added as Easy level AI-controlled player (classic).
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/srv_main.cpp:2493) - AI*4 has been added as Easy level AI-controlled player (classic).
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/srv_main.cpp:2493) - AI*5 has been added as Easy level AI-controlled player (classic).
[info] freeciv21-server (C:/Tools/msys64/home/James/Freeciv21/server/server.cpp:762) - Now accepting new client connections on port 5556.

For introductory help, type 'help'.

However the tests fail

Start testing: Jan 01 19:00 Central Standard Time
----------------------------------------------------------
1/3 Testing: test_utility_paths
1/3 Test: test_utility_paths
Command: "C:/Tools/msys64/home/James/Freeciv21/build/Debug/test_utility_paths.exe"
Directory: C:/Tools/msys64/home/James/Freeciv21/build/utility/tests
"test_utility_paths" start time: Jan 01 19:00 Central Standard Time
Output:
----------------------------------------------------------
********* Start testing of test_paths *********
Config: Using QtTest library 5.15.11, Qt 5.15.11 (x86_64-little_endian-llp64 shared (dynamic) release build; by GCC 13.2.0), windows 10
PASS   : test_paths::initTestCase()
PASS   : test_paths::is_safe_filename()
PASS   : test_paths::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted, 1ms
********* Finished testing of test_paths *********
<end of output>
Test time =   0.11 sec
----------------------------------------------------------
Test Passed.
"test_utility_paths" end time: Jan 01 19:00 Central Standard Time
"test_utility_paths" time elapsed: 00:00:00
----------------------------------------------------------

2/3 Testing: test_rulesets
2/3 Test: test_rulesets
Command: "C:/Tools/msys64/home/James/Freeciv21/build/Debug/test_rulesets.exe"
Directory: C:/Tools/msys64/home/James/Freeciv21
"test_rulesets" start time: Jan 01 19:00 Central Standard Time
Output:
----------------------------------------------------------
********* Start testing of test_rulesets *********
Config: Using QtTest library 5.15.11, Qt 5.15.11 (x86_64-little_endian-llp64 shared (dynamic) release build; by GCC 13.2.0), windows 10
PASS   : test_rulesets::initTestCase()
FAIL!  : test_rulesets::load(default) Compared values are not the same
   Actual   (p.state())           : Running
   Expected (QProcess::NotRunning): NotRunning
C:/Tools/msys64/home/James/Freeciv21/build/data/tests/Debug/rulesets.cpp(52) : failure location
QWARN  : test_rulesets::load(default) QProcess: Destroyed while process ("C:\\Tools\\msys64\\home\\James\\Freeciv21\\build\\Debug\\freeciv21-server.exe") is still running.
PASS   : test_rulesets::load(civ1)
PASS   : test_rulesets::load(civ2)
FAIL!  : test_rulesets::load(civ2civ3) Compared values are not the same
   Actual   (p.state())           : Running
   Expected (QProcess::NotRunning): NotRunning
C:/Tools/msys64/home/James/Freeciv21/build/data/tests/Debug/rulesets.cpp(52) : failure location
QWARN  : test_rulesets::load(civ2civ3) QProcess: Destroyed while process ("C:\\Tools\\msys64\\home\\James\\Freeciv21\\build\\Debug\\freeciv21-server.exe") is still running.
PASS   : test_rulesets::load(classic)
FAIL!  : test_rulesets::load(experimental) Compared values are not the same
   Actual   (p.state())           : Running
   Expected (QProcess::NotRunning): NotRunning
C:/Tools/msys64/home/James/Freeciv21/build/data/tests/Debug/rulesets.cpp(52) : failure location
QWARN  : test_rulesets::load(experimental) QProcess: Destroyed while process ("C:\\Tools\\msys64\\home\\James\\Freeciv21\\build\\Debug\\freeciv21-server.exe") is still running.
FAIL!  : test_rulesets::load(multiplayer) Compared values are not the same
   Actual   (p.state())           : Running
   Expected (QProcess::NotRunning): NotRunning
C:/Tools/msys64/home/James/Freeciv21/build/data/tests/Debug/rulesets.cpp(52) : failure location
QWARN  : test_rulesets::load(multiplayer) QProcess: Destroyed while process ("C:\\Tools\\msys64\\home\\James\\Freeciv21\\build\\Debug\\freeciv21-server.exe") is still running.
FAIL!  : test_rulesets::load(royale) Compared values are not the same
   Actual   (p.state())           : Running
   Expected (QProcess::NotRunning): NotRunning
C:/Tools/msys64/home/James/Freeciv21/build/data/tests/Debug/rulesets.cpp(52) : failure location
QWARN  : test_rulesets::load(royale) QProcess: Destroyed while process ("C:\\Tools\\msys64\\home\\James\\Freeciv21\\build\\Debug\\freeciv21-server.exe") is still running.
PASS   : test_rulesets::load(stub)
PASS   : test_rulesets::load(error)
PASS   : test_rulesets::cleanupTestCase()
Totals: 7 passed, 5 failed, 0 skipped, 0 blacklisted, 42100ms
********* Finished testing of test_rulesets *********
<end of output>
Test time =  42.17 sec
----------------------------------------------------------
Test Failed.
"test_rulesets" end time: Jan 01 19:00 Central Standard Time
"test_rulesets" time elapsed: 00:00:42
----------------------------------------------------------

3/3 Testing: test_server_cli
3/3 Test: test_server_cli
Command: "C:/Tools/msys64/home/James/Freeciv21/build/Debug/test_server_cli.exe"
Directory: C:/Tools/msys64/home/James/Freeciv21
"test_server_cli" start time: Jan 01 19:00 Central Standard Time
Output:
----------------------------------------------------------
********* Start testing of test_cli *********
Config: Using QtTest library 5.15.11, Qt 5.15.11 (x86_64-little_endian-llp64 shared (dynamic) release build; by GCC 13.2.0), windows 10
PASS   : test_cli::initTestCase()
PASS   : test_cli::read(empty)
PASS   : test_cli::read(comment)
PASS   : test_cli::read(set)
FAIL!  : test_cli::read(reread) Compared values are not the same
   Actual   (p.state())           : Running
   Expected (QProcess::NotRunning): NotRunning
C:/Tools/msys64/home/James/Freeciv21/build/server/tests/Debug/cli.cpp(55) : failure location
QWARN  : test_cli::read(reread) QProcess: Destroyed while process ("C:\\Tools\\msys64\\home\\James\\Freeciv21\\build\\Debug\\freeciv21-server.exe") is still running.
PASS   : test_cli::read(unknown)
PASS   : test_cli::cleanupTestCase()
Totals: 6 passed, 1 failed, 0 skipped, 0 blacklisted, 18396ms
********* Finished testing of test_cli *********
<end of output>
Test time =  18.51 sec
----------------------------------------------------------
Test Failed.
"test_server_cli" end time: Jan 01 19:01 Central Standard Time
"test_server_cli" time elapsed: 00:00:18
----------------------------------------------------------

End testing: Jan 01 19:01 Central Standard Time
jwrober commented 9 months ago

Example output using Visual Studio (Windows Clang)

Test project C:/Users/James/Source/Repos/freeciv21/build-vs
    Start 1: test_utility_paths
1/3 Test #1: test_utility_paths ...............   Passed    0.54 sec
    Start 2: test_rulesets
2/3 Test #2: test_rulesets ....................***Failed   54.18 sec
    Start 3: test_server_cli
3/3 Test #3: test_server_cli ..................***Failed   27.34 sec
33% tests passed, 2 tests failed out of 3
Total Test time (real) =  82.35 sec
The following tests FAILED:
      2 - test_rulesets (Failed)
      3 - test_server_cli (Failed)
Errors while running CTest
Output from these tests are in: C:/Users/James/Source/Repos/freeciv21/build-vs/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
Start testing: Jan 01 19:41 Central Standard Time
----------------------------------------------------------
1/3 Testing: test_utility_paths
1/3 Test: test_utility_paths
Command: "C:/Users/James/Source/Repos/freeciv21/build-vs/Debug/test_utility_paths.exe"
Directory: C:/Users/James/Source/Repos/freeciv21/build-vs/utility/tests
"test_utility_paths" start time: Jan 01 19:41 Central Standard Time
Output:
----------------------------------------------------------
<end of output>
Test time =   0.54 sec
----------------------------------------------------------
Test Passed.
"test_utility_paths" end time: Jan 01 19:41 Central Standard Time
"test_utility_paths" time elapsed: 00:00:00
----------------------------------------------------------

2/3 Testing: test_rulesets
2/3 Test: test_rulesets
Command: "C:/Users/James/Source/Repos/freeciv21/build-vs/Debug/test_rulesets.exe"
Directory: C:/Users/James/Source/Repos/freeciv21
"test_rulesets" start time: Jan 01 19:41 Central Standard Time
Output:
----------------------------------------------------------
<end of output>
Test time =  54.18 sec
----------------------------------------------------------
Test Failed.
"test_rulesets" end time: Jan 01 19:42 Central Standard Time
"test_rulesets" time elapsed: 00:00:54
----------------------------------------------------------

3/3 Testing: test_server_cli
3/3 Test: test_server_cli
Command: "C:/Users/James/Source/Repos/freeciv21/build-vs/Debug/test_server_cli.exe"
Directory: C:/Users/James/Source/Repos/freeciv21
"test_server_cli" start time: Jan 01 19:42 Central Standard Time
Output:
----------------------------------------------------------
<end of output>
Test time =  27.34 sec
----------------------------------------------------------
Test Failed.
"test_server_cli" end time: Jan 01 19:42 Central Standard Time
"test_server_cli" time elapsed: 00:00:27
----------------------------------------------------------

End testing: Jan 01 19:42 Central Standard Time