aws-actions / aws-devicefarm-mobile-device-testing

Run automated Mobile Device Testing on AWS Device Farm
https://docs.aws.amazon.com/devicefarm/latest/developerguide/welcome.html#automated-test-intro
MIT No Attribution
5 stars 1 forks source link

Add support for overriding run-settings #1

Closed mina-kleid closed 1 year ago

mina-kleid commented 1 year ago

I have started using this action for running tests on AWS DeviceFarm and it is working great. Though i found providing the run-settings only as a path to a json file limiting.

Use case

I want to schedule the runs to run on different device pools

Expected

I can easily provide a matrix with different pools or have different workflows that use different devicePoolArn while having the same run-settings

Actual

I need to have a different file for each device pool

Suggested solution

  1. Add support to provide inputs to the action that override the one from the run-settings file
  2. Add support to provide run-settings as JSON instead of a file path. This way there is a possibility to generate different JSON for different settings
gmazzo commented 1 year ago

Actually, it would be nice if you could honor GitHub Action design and best practices by not relying on an external file to configure the action, and rather using inputs parameters instead.

danjhd commented 1 year ago

This is good idea however since GH Action inputs only allow basic types and not object or array types moving the complexity of the json file to inputs is not easily done whilst still maintaining a simple user experience. Perhaps a compromise would be to do as @mina-kleid suggests and pick specific settings like the device pool arn that can be provided as optional inputs and if they are provided as inputs they override the contents of the file?

mina-kleid commented 1 year ago

@danjhd Thanks for your answer.

I believe having the option to add inputs will give more flexibility. For our specific use case devicePoolArn & projectArn would be great as a start.

On the other hand as @gmazzo mentioned there will always be different use cases that will require having other attributes so might be a good idea to find a way to have all the attributes through inputs to solve this limitation.

Might be a good start to have a first iteration with the basic inputs that have STRING_VALUE to be as inputs like devicePoolArn, projectArn, appArn, networkProfileArn etc.

danjhd commented 1 year ago

@mina-kleid @gmazzo Thank you for your feedback. Please review the pre-release here:

https://github.com/aws-actions/aws-devicefarm-mobile-device-testing/releases/tag/v2-beta

Hopefully these changes meet your expectations but also keep the action as generic and flexible for the future as possible.

Please give feedback, also please note the mention in the changelog that this release will be a breaking change. As @gmazzo pointed out it makes more sense to be closer to GH best practices and not reference a file.

mina-kleid commented 1 year ago

@danjhd Thanks for the fast response and changes. Looks good for solving this issue & #2

danjhd commented 1 year ago

Closing this with release of v2.0