** Introduction This repository provides actions to interact with AWS devicefarms, implemented using the [[https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DeviceFarm.html][AWS JavaScript SDK.]] AWS credentials need to be configured using the [[https://github.com/aws-actions/configure-aws-credentials][configure AWS credentials action]].
Due to the nature of the application this was developed for the following restrictions apply for now:
[[https://github.com/realm/aws-devicefarm-sample-data][Sample test data and applications]] simulating this case are provided in a separate repository.
Two types of actions are provided: Simple API wrappers - useful for debugging, testing or building workflows otherwise not properly supported - and complex actions, wrapping several API calls. Typically complex actions should be used, whenever possible.
*** Examples After adding secrets for AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEVICE_POOL_ARN and AWS_PROJECT_ARN (see AWS setup section below if no devicefarm is configured) the following workflow will create a test run with a mix of downloaded and local data, and wait for it to execute. You may want to adjust =app_file=, =test_spec_file= and =test_package_file= before running it - as shown it will pull demo data for the run:
**** Example with remote sources and artifact pulling
on: push: branches: [ master ] pull_request: branches: [ master ]
name: Test application on device farm
jobs: deploy: name: Deploy runs-on: ubuntu-latest environment: production
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Schedule test run with mixed data
uses: realm/aws-devicefarm/test-application@master
with:
name: schedule-run
file_artifacts: |
Test spec file.yml
Customer Artifacts.zip
project_arn: ${{ secrets.AWS_PROJECT_ARN }}
device_pool_arn: ${{ secrets.AWS_DEVICE_POOL_ARN }}
app_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/fi.aardsoft.devicefarmsample-debug.apk
app_type: ANDROID_APP
test_type: APPIUM_PYTHON
test_package_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/sample_env_python3.zip
test_package_type: APPIUM_PYTHON_TEST_PACKAGE
test_spec_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/test_spec.yaml
test_spec_type: APPIUM_PYTHON_TEST_SPEC
remote_src: true
**** Example with remote sources, inline test specification and artifact pulling It is also possible to have the test specification inline:
on: push: branches: [ master ] pull_request: branches: [ master ]
name: Test application on device farm
jobs: deploy: name: Deploy runs-on: ubuntu-latest environment: production
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Schedule test run with mixed data
uses: realm/aws-devicefarm/test-application@master
with:
name: schedule-run-inline
file_artifacts: |
Test spec file.yml
Customer Artifacts.zip
project_arn: ${{ secrets.AWS_PROJECT_ARN }}
device_pool_arn: ${{ secrets.AWS_DEVICE_POOL_ARN }}
app_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/fi.aardsoft.devicefarmsample-debug.apk
app_type: ANDROID_APP
test_type: APPIUM_PYTHON
test_package_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/sample_env_python3.zip
test_package_type: APPIUM_PYTHON_TEST_PACKAGE
test_spec_file: test_spec.yaml
test_spec_type: APPIUM_PYTHON_TEST_SPEC
test_spec: |
version: 0.1
phases:
install:
commands:
- export PYTHON_VERSION=3
pre_test:
commands:
- adb -s $DEVICEFARM_DEVICE_UDID shell pm grant fi.aardsoft.devicefarmsample android.permission.READ_EXTERNAL_STORAGE
- adb -s $DEVICEFARM_DEVICE_UDID shell pm grant fi.aardsoft.devicefarmsample android.permission.WRITE_EXTERNAL_STORAGE
test:
commands:
- adb -s $DEVICEFARM_DEVICE_UDID shell am instrument -w -r -e foo bar -e bar baz fi.aardsoft.devicefarmsample
post_test:
commands:
- adb -s $DEVICEFARM_DEVICE_UDID pull /storage/emulated/0/Android/data/fi.aardsoft.devicefarmsample/files/output.txt
- mv output.txt $DEVICEFARM_LOG_DIR/test-output.txt
artifacts:
- $DEVICEFARM_LOG_DIR
remote_src: true
** Available actions *** API wrappers The following actions are just wrappers around the [[https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DeviceFarm.html][JavaScript SDK]]. In addition to the JSON returned by the API they also return the JSON fields most likely to be useful for later calls - saving some JSON parsing compared to building actions on top of the AWS CLI. For each action documentation links to both the API and the matching call in AWS CLI are provided - with the latter typically being the more useful when building a workflow on top of those actions without trying to touch the JavaScript.
**** create-upload Create an upload for a file named =name=, of type =type=, in the project specified by =project_arn=. Note that a file needs to be uploaded to the URL from the result in a second step - for an action handling the upload as well check =upload-file=.
***** inputs
***** outputs
***** documentation
***** usage example
A step similar to the following could be used to upload the actual file:
The status of the upload should be checked afterwards with =get-upload=, possibly in a loop until the status changes.
**** delete-upload Deletes a resource uploaded to a devicefarm project, specified by =resource_arn=.
***** inputs
***** outputs This method does not return data, and completes successfully even if the resource requested for deletion does not exist.
***** documentation
***** usage example
**** get-device-pool Get information about a specific device pool.
***** inputs
***** outputs
***** documentation
***** usage example
**** get-project Return details of the given project.
***** inputs
***** outputs
***** documentation
***** usage example
**** get-run Return details for a specific test run.
***** inputs
***** outputs
***** documentation
***** usage example Typically the ARN is obtained from an earlier step:
**** get-upload Return details for the upload specified by =resource_arn=.
***** inputs
***** outputs
***** documentation
***** usage example
**** list-artifacts List artifacts for a resource. Note that you can only list one artifact type at one time, and can't query specific artifact names. To retrieve a specific artifact loop over the returned array, check for artifact names, and GET the included URL to retrieve the artifact.
***** inputs
***** outputs
***** documentation
***** usage example
**** list-device-pools Return a list of device pools configured for the specified project.
***** inputs
***** outputs
***** documentation
***** usage example
**** list-runs List test runs in the specified device farm project.
***** inputs
***** outputs
***** documentation
***** usage example
**** list-uploads Return all uploads in the project specified by =project_arn= as a JSON string.
***** inputs
***** outputs
***** documentation
***** usage example
The output can be referenced in following actions using the step ID:
with: input: ${{ steps.list-uploads.outputs.data }}
**** schedule-run Schedule a test run with resources uploaded to AWS already. This action returns directly after scheduling a run - this behaviour is useful to avoid blocking a workflow if other steps can still be executed, but requires later checking with =get-run= if the run has finished. For an action capable of uploading required files as well as blocking until a test run has finished see =test-application=.
When called without a test specification the run will be triggered with the default test environment of the specified test type.
***** inputs
***** outputs
***** documentation
***** usage example This example references ARNs obtained from previous upload steps:
* Complex actions ** download-artifacts This action downloads one or more artifacts from a test run. Trying to download a non-existent artifact will log a warning and omit the file frem the output, but not abort.
***** inputs
***** outputs
***** usage example
This assumes the run with id =schedule-run= created customer artifacts:
The returned JSON looks like this - note the missing invalid file:
{ "FILE": [ "Test spec file.yml", "Customer Artifacts.zip" ], "SCREENSHOT": [ ], "LOG": [ ] }
**** upload-file This action creates a file upload and then uploads a file.
***** inputs
***** outputs
***** usage example
**** test-application This action schedules a test run and waits for the result. It can either use already uploaded files, or upload local or remote files.
When called without a test specification the run will be triggered with the default test environment of the specified test type.
***** inputs
***** outputs
***** usage example This example pulls remote sources, and uploads them to a device farm before scheduling a run:
** Setting up AWS *** Create a test project In the [[https://us-west-2.console.aws.amazon.com/devicefarm/home][Device Farm console]] create a new project, and copy the displayed ARN - this is the project ARN required by some actions. Next go to =Project settings=, =Device pools= and create a new pool. Retrieve the pool ARN using the [[https://docs.aws.amazon.com/cli/latest/reference/devicefarm/list-device-pools.html][list-device-pools]] CLI command, or using the list-device-pools action if AWS CLI is not set up.
Secrets named AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEVICE_POOL_ARN and AWS_PROJECT_ARN or similar need to be set using the ARNs just created and account secrets.
It is recommended to use a dedicated role user for devicefarm access. the AWS role to limit access to devicefarm only is =AWSDeviceFarmFullAccess=.
** Development *** Contributing A pull request should also include build results, built with the [[https://github.com/realm/aws-devicefarm/blob/master/.github/workflows/integration-test.yml#L29][same node version as used in CI]] (at the time of writing this: 16):
$ npm ci [...] $ npm run package
aws-devicefarm@1.0.0 package node tools/build.js
Building create-upload [...]
[[https://github.com/nvm-sh/nvm][nvm]] is an easy way to add additional node versions to the local machine. After following the instructions there:
$ nvm install 16 Downloading and installing node v16.20.2... Downloading https://nodejs.org/dist/v16.20.2/node-v16.20.2-linux-x64.tar.xz... --2023-10-02 12:00:42-- https://nodejs.org/dist/v16.20.2/node-v16.20.2-linux-x64.tar.xz Resolving nodejs.org (nodejs.org)... 104.20.22.46, 104.20.23.46, 2606:4700:10::6814:162e, ... Connecting to nodejs.org (nodejs.org)|104.20.22.46|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 22556484 (22M) [application/x-xz] Saving to: ‘/home/user/.nvm/.cache/bin/node-v16.20.2-linux-x64/node-v16.20.2-linux-x64.tar.xz’
/home/user/.nvm/.cache 100%[======================================>] 21.51M 2.70MB/s in 8.0s
2023-10-02 12:00:50 (2.71 MB/s) - ‘/home/user/.nvm/.cache/bin/node-v16.20.2-linux-x64/node-v16.20.2-linux-x64.tar.xz’ saved [22556484/22556484]
Computing checksum with sha256sum Checksums matched! Now using node v16.20.2 (npm v8.19.4)
*** Local testing Local testing is possible to some extend using [[https://github.com/nektos/act][act]]. All secrets should be exported as environment variable, otherwise act will prompt for it:
$ act -s AWS_ACCESS_KEY_ID -s AWS_SECRET_ACCESS_KEY -s AWS_DEVICE_POOL_ARN -s AWS_PROJECT_ARN