CI Integration (wait for test results in the gradle plugin)

[danielesegato]

The current behavior of the plugin is uploading the artifact on the AWS device farm and then provide an URL to follow it.

For a CI system this is not what you want: you want the test to WAIT to complete and fail if any test has failed / succeed if all test failed.

I'd like to have a task like: devicefarmUploadAndWait that makes the gradle script stop and wait for the result (possibly sending multiple modules projects in parallel and wait for all of them).

Without this I can't use AWS Device Farm for my CI.

[andreschab90]

It would be awesome, if the gradle task would wait for the test results

[paulo-raca]

+1 -- Is anyone working on that?

[andreschab90]

i don't think so. i don't need this, because i wrote an own script for my ci server

[andreas-marschke]

I got it working including exporting ADF results as JUnit XML. If anyone is interested I can post an MR here.

CC @appwiz

[mahyarv]

@andreas-marschke Can you please post?

[andreas-marschke]

@andreschab90 @paulo-raca @danielesegato @mahyarv #22

[danielesegato]

@andreas-marschke I've no time to test it right now. But if I did i would appreciate some instruction along with your MR, I guess it may benefit the others as well.

[andreas-marschke]

@danielesegato See

https://github.com/awslabs/aws-device-farm-gradle-plugin/pull/22/files#diff-04c6e90faac2675aa89e2176d2eec7d8R77

For instructions.

Just add this to your devicefarm closure to enable waiting

useWait()

And if you wish to store test results:

useOutput("JUnit")
setTestsDestination("build/tests/adf-results")

Thats it.

[erickentry]

I did try the MR by running 1 test on 3 devices on ADF (1 fail, 2 success). But it seems like the XMLs created were not what I expected... 3 files were created, one of them is:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" failures="0" name="my.app.package.name.testClass" tests="1">
  <testcase classname="test" name="test">
  </testcase>
</testsuite>

What did I do wrong?

This is my first time commenting on GitHub issue, apology if I break any code of conduct

@andreas-marschke what you did were awesome :+1:

[andreas-marschke]

@erickentry Was the test class named test and the test also named test?

If my understanding of how your configuration is supposed to work is correct all 3 runs created their individual run results were stored in these files. Can you attache the files here?

I apologize if my implementation does not support multi-device runs as that was not part of my scope/goal-post when I needed to implement it.

Feel free to file an issue and assign me, worst case. Otherwise the PR is still open, you can review the really simple code that I've written for this to implement it.

[erickentry]

@andreas-marschke below were the files generated TEST.zip

Test class name  : ConecdreamChooseCategory
Test method name : test

I'll try to implement the XML creation for multi-device runs. If the PR is still open, I'll review. But if it is merged, I'll PR to master :)

Thank you so much...

[igor-brishkoski]

Any updates here? It would be really helpful instead writing custom scripts.