search

adamrehn / ue4cli

Command-line interface for Unreal Engine 4
https://docs.adamrehn.com/ue4cli/
MIT License
249 stars 45 forks source link

`ue4 test` always exits with non-zero in UE 4.27 #39

Closed jonpas closed 2 years ago

jonpas commented 2 years ago

Seems like 4.27 changed the test exit code behaviour. In 4.26 it always exited with RequestExit(1), but in 4.27 it's doing RequestExitWithStatus(1, 0).

ue4 test in 4.26 returns the correct exit code, while 4.27 always fails, which is a problem for CI use.

UE 4.26

[2021.09.13-10.30.15:828][420]LogAutomationController: Display: Test Completed. Result={Passed} Name={FunctionalTest1} Path={Project.Functional Tests./Game/FirstPersonCPP/Maps/FTestMap.FunctionalTest1}
[2021.09.13-10.30.15:835][420]LogAutomationController: BeginEvents: Project.Functional Tests./Game/FirstPersonCPP/Maps/FTestMap.FunctionalTest1
[2021.09.13-10.30.15:846][420]LogAutomationController: FunctionalTest_1: Assertion passed (Version is not empty)
[2021.09.13-10.30.15:858][420]LogAutomationController: FunctionalTest_1: Assertion passed (Version component is numeric)
[2021.09.13-10.30.15:869][420]LogAutomationController: FunctionalTest_1: Assertion passed (Version component is numeric)
[2021.09.13-10.30.15:880][420]LogAutomationController: FunctionalTest_1: Assertion passed (Version component is numeric)
[2021.09.13-10.30.15:892][420]LogAutomationController: EndEvents: Project.Functional Tests./Game/FirstPersonCPP/Maps/FTestMap.FunctionalTest1
[2021.09.13-10.30.15:904][420]LogAutomationController: Display: Report can be opened in the editor at 'D:/Work/IDI/internal/basic/Saved/Automation/Reports/'
[2021.09.13-10.30.15:915][422]LogAutomationCommandLine: Forcing shutdown.
[2021.09.13-10.30.15:926][422]LogAutomationCommandLine: Display: **** TEST COMPLETE. EXIT CODE: 0 ****
[2021.09.13-10.30.15:938][422]LogWindows: FPlatformMisc::RequestExit(1)
[2021.09.13-10.30.15:950][422]LogCore: Engine exit requested (reason: Win RequestExit)

UE 4.27

[2021.09.13-10.26.17:703][436]LogAutomationController: Display: Test Completed. Result={Passed} Name={FunctionalTest1} Path={Project.Functional Tests./Game/FirstPersonCPP/Maps/FTestMap.FunctionalTest1}
[2021.09.13-10.26.17:712][436]LogAutomationController: BeginEvents: Project.Functional Tests./Game/FirstPersonCPP/Maps/FTestMap.FunctionalTest1
[2021.09.13-10.26.17:726][436]LogAutomationController: FunctionalTest_1: Assertion passed (Version is not empty)
[2021.09.13-10.26.17:741][436]LogAutomationController: FunctionalTest_1: Assertion passed (Version component is numeric)
[2021.09.13-10.26.17:756][436]LogAutomationController: FunctionalTest_1: Assertion passed (Version component is numeric)
[2021.09.13-10.26.17:767][436]LogAutomationController: FunctionalTest_1: Assertion passed (Version component is numeric)
[2021.09.13-10.26.17:779][436]LogAutomationController: FunctionalTest_1: 0.1.0
[2021.09.13-10.26.17:790][436]LogAutomationController: EndEvents: Project.Functional Tests./Game/FirstPersonCPP/Maps/FTestMap.FunctionalTest1
[2021.09.13-10.26.17:803][436]LogAutomationController: Display: Report can be opened in the editor at 'D:/Work/IDI/internal/basic/Saved/Automation/Reports/'
[2021.09.13-10.26.17:814][438]LogAutomationCommandLine: Shutting down. GIsCriticalError=0
[2021.09.13-10.26.17:825][438]LogAutomationCommandLine: Display: **** TEST COMPLETE. EXIT CODE: 0 ****
[2021.09.13-10.26.17:836][438]LogWindows: FPlatformMisc::RequestExitWithStatus(1, 0)
[2021.09.13-10.26.17:848][438]LogCore: Engine exit requested (reason: Win RequestExit)
jonpas commented 2 years ago

Still the same on 4.27.1.