DaedalicEntertainment / ue4-test-automation

Facilitates setting up integration test suits with Unreal Engine 4 Gauntlet.
https://www.daedalic.com
MIT License
215 stars 62 forks source link

Tests fail when run from Jenkins #35

Open dcordovag opened 2 years ago

dcordovag commented 2 years ago

Hello! I successfully managed to integrate the plugin in my 5.0 project and now i'm looking into running the tests through Jenkins.

Running from the command line works fine but I get an error related with the 3d device when trying with Jenkins, I'm assuming it's because the Jenkins process runs without an active window:

LogD3D12RHI: Error: hr failed 
 at D:\Projects\MyProject\Engine\Source\Runtime\D3D12RHI\Private\Windows\WindowsD3D12Viewport.cpp:224 
 with error DXGI_ERROR_NOT_CURRENTLY_AVAILABLE

Is there anything i'm missing or it's just not possible to do this with the plugin at the time?

Cheers!

dcordovag commented 2 years ago

Finally could make it work. The right command is below, with emphasis in the -unattended and -NullRHI params:

RunUAT.bat RunUnreal -project="MyProject.uproject" -scriptdir="D:/Projects/MyProject" -platform="Win64" -configuration="Development" -build="editor" -test="DaedalicTestAutomationPlugin.Automation.DaeGauntletTest(JUnitReportPath=D:/Reports/junit-report.xml,ReportPath=D:/Reports)" -verbose -unattended -NullRHI

My only question now is how to confirm if the Performance test is accurate since no window is instantiated. I see the test taking a long time, which would indicate that it ran correctly, but that's all the info i have to go by

Cheers!

SalahAdDin commented 2 years ago

Hello! I successfully managed to integrate the plugin in my 5.0 project and now i'm looking into running the tests through Jenkins.

Running from the command line works fine but I get an error related with the 3d device when trying with Jenkins, I'm assuming it's because the Jenkins process runs without an active window:

LogD3D12RHI: Error: hr failed 
 at D:\Projects\MyProject\Engine\Source\Runtime\D3D12RHI\Private\Windows\WindowsD3D12Viewport.cpp:224 
 with error DXGI_ERROR_NOT_CURRENTLY_AVAILABLE

Is there anything I'm missing or it's just not possible to do this with the plugin at the time?

Cheers!

Does this work with the last Unreal Engine version?

thomrohit commented 2 years ago

@dcordovag Could you let us know how you got it to work on UE 5 ? where the plugin need to be placed location and if you had to compile the code

dcordovag commented 2 years ago

Does this work with the last Unreal Engine version?

Yes! there's a 5.0 branch here, that's the one i'm actually using

@dcordovag Could you let us know how you got it to work on UE 5 ? where the plugin need to be placed location and if you had to compile the code

I followed the same instructions in the readme file. You need the Engine's source code but besides that it should work fine. What trouble did you have?