DaedalicEntertainment / ue4-test-automation

Facilitates setting up integration test suits with Unreal Engine 4 Gauntlet.
https://www.daedalic.com
MIT License
215 stars 62 forks source link

Help with parameterized tests #11

Open tyler-pugmire opened 3 years ago

tyler-pugmire commented 3 years ago

Hello, my team is looking to use this plugin for automated tests. I've been looking at the parameterized tests and may be misunderstanding how to use them.

I'll use a movement test as an example of how I picture them working: BP_MoveTest: The test blueprint will apply InputAxis based on parameters provided and use a trigger box to confirm. BP_MoveParams: Base class for parameters inherited from Object. BP_MoveForwardParams: Inherits from BP_MoveParams providing correct parameters to test moving forward. This would be a parameter added to an instance of BP_MoveTest BP_MoveRightParams: Similar to BP_MoveForwardParams but for moving right

After trying it this way it appears to not be correct as the parameters passed to the test won't be valid. Your examples use 2 different blueprint classes both inherited from Object, but only use display name as a difference between the tests. I am struggling to figure out how you would retrieve data from the parameter object.

It's been a while since I've used Unreal so probably something obvious that I'm missing. Thanks for any help.

npruehs commented 3 years ago

Hey @tyler-pugmire.

are you able to cast the parameter? Or is the parameter invalid immediately at the beginning of Act?

Also, can you confirm that the example parametrized tests of the plugin are working (just to rule out a regression in the plugin itself)? You might need to enable "Show plugin content" for your content browser to see them.

tyler-pugmire commented 3 years ago

Yeah the example tests work as far as I can tell.

Running the tests again this morning the parameters are valid just can't be casted back to original type.

BP_Move

Using this test I tried a couple different parameter types.

Move_Parameters

Both of these are child blueprints of BP_Move_Params which inherits from Object. With these params the output of the test looked like

LogDaeTest: ADaeTestSuiteActor::RunNextTest - Test: BP_Move - BP_MoveForwardParams
LogBlueprintUserMessages: [BP_Move] BP_MoveForwardParams
LogBlueprintUserMessages: [BP_Move] Cast Failed
LogDaeTest: ADaeTestSuiteActor::OnTestSuccessful - Test: BP_Move - BP_MoveForwardParams
LogDaeTest: ADaeTestSuiteActor::RunNextTest - Test: BP_Move - BP_MoveRightParams
LogBlueprintUserMessages: [BP_Move] BP_MoveRightParams
LogBlueprintUserMessages: [BP_Move] Cast Failed
LogDaeTest: ADaeTestSuiteActor::OnTestSuccessful - Test: BP_Move - BP_MoveRightParams

MoveParametersSingle

I then tried with just BP_Move_Params which inherits from Object and got the following output.

LogDaeTest: ADaeTestSuiteActor::RunAllTests - Test Suite: BP_BVT_SUITE_2
LogDaeTest: ADaeTestSuiteActor::RunNextTest - Test: BP_Move - BP_Move_Params
LogBlueprintUserMessages: [BP_Move] BP_Move_Params
LogBlueprintUserMessages: [BP_Move] Cast Failed
LogDaeTest: ADaeTestSuiteActor::OnTestSuccessful - Test: BP_Move - BP_Move_Params
shwarnock commented 3 years ago

I too am having the same issue, though I believe it is two separate issues.

The first issue is that, when I load up the editor, if I already had my parameters set in my test, the Parameter.IsValid check always fails and returns nullptr. If I remove my parameter and add it back then run the test, the parameter loads fine.

The second half of this is that I do not seem to be able to cast my parameters UObject to my Parameter subclass. It seems that the parameter reference is not a UObject reference but actually a UBlueprint reference.

realfleo commented 1 year ago

I am having the same casting problems. Did you find solutions? Thanks.

realfleo commented 1 year ago

Ok for the parameter casting problem I found a great article explaining why.

https://1danielcoelho.github.io/unreal-engine-basics-base-classes/

As a solution I created a function that gets the CDO from the GeneratedClass to access the "value" members via GetValue(). Note that for me UStringObject is just a wrapper over FString.

FString ADaeTestActor::GetStringValueFromParameter(UObject loadedBP) { FString returnString; if (UBlueprint bp = Cast(loadedBP)) { UClass parentClass = bp->ParentClass; if (UStringObject::StaticClass() == parentClass) {
if (UClass
generatedClass = bp->GeneratedClass) { returnString = generatedClass->GetDefaultObject()->GetValue(); } } } return returnString; }