changes to support test sharding of test cases across devices in a device pool

[anandkiran2007]

Notes PROBLEM STATEMENT: In certain situations, consumers prioritize obtaining faster test execution results over running the same set of tests across different devices.

SOLUTION: This change aims to tackle the aforementioned issue by introducing additional arguments for users, namely splitTestsAcrossDevices and testSpecFiles (under DeviceState). With this enhancement, the configuration will be modified as follows: [Rephrase the configuration example based on the provided information.]

devicefarm {

    // Required. The Project must already exist. You can create a project in the AWS console.
    projectName "EspressoTests" // required: Must already exist.

    // Optional. Defaults to "Top Devices"
    devicePool "AOSP Devices"

    // Optional. Default is 150 minutes
    executionTimeoutMinutes 20

    // Optional. Set to "off" if you want to disable device video recording during a run. Default is "on"
    videoRecording "on"

    // Optional. Set to "off" if you want to disable device performance monitoring during a run. Default is "on"
    performanceMonitoring "on"

    // Required. You must specify either accessKey and secretKey OR roleArn. roleArn takes precedence.
    authentication {
        roleArn "xxxx"
    }

    // Optional block. Radios default to 'on' state, all parameters are optional
    devicestate {
        extraDataZipFile null // or ‘null’ if you have no extra data. Default is null.
        auxiliaryApps files(file("src/androidTest/auxiliaryApps/orchestrator-1.2.0.apk"),
                file("src/androidTest/auxiliaryApps/test-services-1.2.0.apk"),
                file("src/androidTest/auxiliaryApps/CDPAutomationHelperApp.apk")) // or ‘files()’ if you have no auxiliary apps. Default is an empty list.
        testSpecFiles files(file("src/androidTest/yaml/EURegionNewUserTests.yml"),
                file("src/androidTest/yaml/Core_Playback_People_Places_Tests.yml"),
                file("src/androidTest/yaml/JPRegionNewUserTests.yml"),
                file("src/androidTest/yaml/NewUserScenarios.yml"))// or ‘files()’ if you have no auxiliary apps. Default is an empty list.
        wifi "on"
        bluetooth "off"
        gps "off"
        nfc "on"
        latitude 47.6204 // default
        longitude -122.3491 // default
    }

    testShardingEnabled true

    // Instrumentation
    instrumentation {
        // Optional. See the AWS Developer docs for filter rules
    }
}

In the scenario where the user has multiple devices in the device pool, the updated implementation aims to create separate runs for each testSpec File provided by the user. As a result, the user will observe four different runs in the given example. However, it is important to note that the responsibility of parsing the overall test runs and managing the results will still lie with the user. This change is intended to enhance the speed of obtaining test results, empowering the user to implement their own mechanism for parsing and managing the multiple test runs.

PROBLEM STATEMENT 2: We are facing nullPointerException for accessKey even when we provide roleArn.

SOLUTION: The updated version of DeviceFarmClientFactory now allows users to provide only the roleArn, and it will automatically fetch the necessary credentials and create an AWSDeviceFarm client for the plugin, ensuring proper functionality.

Testing Verified the changes in device farm

[matzuk]

It's funny that the very useful functionality implemented in this PR is still not merged or even commented on. @anandkiran2007 good luck man. Just try a better solution for your needs - https://marathonlabs.io

[nikhil-dabhade]

@anandkiran2007 Sorry for the delay on this. I am taking a look at this and will revert soon.

[anandkiran2007]

Can this be closed if the changes can be merged ?