Possibility to leverage Xamarin UITest to automate testing in CI

[ShiroYacha]

@onovotny I am looking for a solution to enable xunit unit test automation in CI platforms such as the Xamarin Mobile Center. I seems that currently I can only run unit tests via MSBuild or post-build schell scripts (and the tests are not carried out in the devices)... After checking the project a bit, I think currently it is not possible to automate this type of testing? As Mobile Center supports Xamarin.UITest, I wonder if it is possible to create UITest for the xunit Android/iOS test applications and integrate the results directly to the output window.

For instance, OOTB, we can create a single test case, simulate the Tap event on "Run Everything" and then retrieve the result of each test and create a summary in the output. A bit more advance way is to create dynamically multiple test cases using something like:

[Test, TestCaseSource("GetTestResults")]
public void ExportTestResults(TestResult testResult)
{
// pass or not depends on the results displayed by the xUnit Device UI
}

static TestResult [] GetTestResults()
{
    return GetAllTestResultsFound();
}

I haven't really tested this but it seems that this can be implemented as an external plugin to support CI testing for xunit.device.

[pellet]

I think writing a simple async\await TCP listener to write out the xunit.XML might be the cleaner option. The tests can be auto started and the app killed after running.

[pellet]

I found this https://github.com/spouliot/Touch.Unit you can use the Touch.Server to write the CI's test to stdout/log and parse it to see if the test failed.

[jaimspok]

Touch.Server from the Touch.Unit repository was built for NUnit tests. Has anyone ran it with XUnit tests?

[pellet]

I think you would have to do some modification to get it to work. I ended up writing some simple code from scratch to listen on the tcp port and parse the incoming data for "Failed: 0" as to wether the tests passed or not. I wrote the "test receiver" and android adb test harness deployment in c# using cake frosting. I might try breaking the test receiver code out into a cake plugin for others to use or chuck some code up in gist or something.

[pellet]

I've created a devices.xunit test receiver as a cake plugin. Here is the repo