Allow custom parameters on test execution

[fhbonfante]

Describe Request

I would like the ability to run my tests with filters, using testFilter, assemblyNames or testCategory from unity test framework command line arguments.

This way I could have distinct workflows where I run different test chunks based on what changed.

Suggested solution

Would be nice to have a parameter on orb like:

      - unity/test:
          name: 'test-linux'
          step-name: 'Check if the tests run and results are uploaded'
          unity-license-var-name: 'UNITY_ENCODED_LICENSE'
          unity-username-var-name: 'UNITY_USERNAME'
          unity-password-var-name: 'UNITY_PASSWORD'
          executor:
            name: 'unity/ubuntu'
            target_platform: 'linux-il2cpp'
            editor_version: '2021.3.1f1'
            resource_class: 'medium'
          project-path: 'Unity2D-Demo-Game-CI-CD/src'
          test-platform: 'playmode'
          test-filter: 'MyNamespace.Something.MyTest;MyNamespace.SomethingElse.MyTest2'
          test-category: 'Unit;Integration;AnythingElse;AnotherCategory'
          assembly-names: 'MyUnitTestAssembly;MyIntegrationAssembly;AnyOtherAssembly'
          context: unity

Note the 3 parameters that currently does not exists: test-filter, test-category and assembly-names

And the command executed by the job would be like:

${UNITY_EXECUTABLE} \
  -projectPath $UNITY_DIR \
  -runTests \
  -testPlatform $TEST_PLATFORM \
  -testResults $UNITY_DIR/$TEST_PLATFORM-results.xml \
  -logFile /dev/stdout \
  -batchmode \
  -nographics \
  -enableCodeCoverage \
  -coverageResultsPath $UNITY_DIR/$TEST_PLATFORM-coverage \
  -coverageOptions "generateAdditionalMetrics;generateHtmlReport;generateHtmlReportHistory;generateBadgeReport;" \
  -debugCodeOptimization
  -testFilter "MyNamespace.Something.MyTest;MyNamespace.SomethingElse.MyTest2"
  -assemblyNames "MyUnitTestAssembly;MyIntegrationAssembly;AnyOtherAssembly"
  -testCategory "Unit;Integration;AnythingElse;AnotherCategory"

Currently the test command is executed hard coded from the shells on each OS. Possibly need to extract this to outside and enable to pass parameters to it.

Additional details

Unity test framework testFilters Unity test framework assemblyNames Unity test framework testCategory

[EricRibeiro]

Thank you for bringing this to our attention!

I wonder if including the custom-parameters parameter in the test job would be a better approach. It will allow users to insert any additional flags they deem necessary. Such a parameter only exists in the build job at the moment.

Thoughts?

[fhbonfante]

Good morning @EricRibeiro ! Thank you for the reply! Custom parameters seens pretty good to me and more flexible! It would be something like this:

 custom-parameters: -testFilter "a;b;c" -assemblyNames "a;b;c" -testCategory "a;b;c"

[EricRibeiro]

@fhbonfante, the latest version of the orb has the new parameter available. Let me know what you think!