Add support for taking screenshots (Android)

[ZakTaccardi]

I'm not seeing any documentation related to taking screenshots on Android with Marathon. What is the right way to do this?

[Malinskiy]

@ZakTaccardi test runner is not really aware of taking screenshots. it's possible to capture screenshots from inside the test itself. There is an experimental branch enabling the allure-android which supports something like:

@Test
fun test() {
    deviceScreenshot("screenshot_name")
}

then marathon picks those screenshots automatically. Unfortunately, there are some issues with the current implementation of allure's sdk on the device that I've forwarded to allure's team. Hopefully, there is a solution for these soon.

Relates to #356

In the meantime, you could still use this framework and at the end of the execution of marathon do the adb pull xxx for your screenshots.

[ZakTaccardi]

Sorry I'm a bit confused - I don't use allure at the moment, and I just want to take a screenshot and have it visible in the marathon report.

Is the only mechanism for this allure?

[Malinskiy]

Sorry for the confusion, let me clarify: there is no mechanism currently in the stable version of marathon.

The only standard-ish way to do this ATM is to use allure, but as I investigated this several months ago, it's not working as expected.

[ZakTaccardi]

Ah. So for people looking to replace a test runner like Spoon with Marathon, I think it's very important to support Screenshots natively, and not rely on Allure for doing so?

Maybe Allure could be one implementation of pulling screenshots? But I think it's important that Marathon supports pulling screenshots natively, so I will update the goal of this ticket to reflect that

[Malinskiy]

I don't believe that such a design is reasonable for Android. Android tests runner is self-contained: it doesn't really require an external watcher. If we introduce a mechanism for tests to ask marathon to execute a take screenshot command, it would be a round-trip operation. The test framework itself has enough permissions to take the screenshots. Allure is just a major framework that supports doing this. A generic feature that we should work on is to include a folder from the device to the test artifacts. This way we can add arbitrary screenshots, logs or network traffic dumps to our test report. But these still should be generated by the test framework unless we have a technical reason to do all of this on the test runner's side.

@tagantroy what are your thoughts about this?

[ZakTaccardi]

What is the difference between the two:

1. test framework (JUnit? Espresso?)
2. test runner (Marathon?)

[ZakTaccardi]

If we introduce a mechanism for tests to ask marathon to execute a take screenshot command, it would be a round-trip operation.

Sorry - I may have mispoke. The Marathon test runner should just pull the test artifacts, which should be made flexible enough to include screenshots/arbitrary files if they were generated during the test. I don't know if this is already supported (sounds like it isn't).

As a nice to have, it would be good for Marathon to provide a client side artifact that allows the test itself to do the following:

• take a screenshot (spoon screenshot docs)
• save a file (spoon save a file docs)

So as a user, the developer experience would be something like:

1. include some android marathon .aar in my tests
2. call Marathon.screenshot(..) in my test

[Malinskiy]

The test runner is responsible for running the tests whereas the content of those tests is on the test framework. I understand why test runner might be responsible for screenshots, but in order to create perfect (in terms of time) screenshots test runner has to be aware of what a particular test actually does and this is the domain of the framework (e.g. espresso/allure/etc), not the test runner.

Pulling a random artifact folder is something worth adding (and the referenced allure branch actually does just that - pull a folder from the device and add the contents as artifacts of a particular test). I don't really like the idea of creating yet another test utility classes when you can already use existing ones like allure. I can also understand that I'm a bit pushing here for a framework that you might not really want in your test code.

@tagantroy @Vacxe what are your thoughts on this?

[Nilzor]

Came looking for this as well. You shouldn't need to depend on any third party libs though:

• Allure-dependency is bad for this use case because it forces consumer onto a whole different DSL.
• Spoon dependency shouldn't be required either just to take screenshots, although the implementation could be used as inspiration

You can simply take screenshots with Espresso (or any custom screenshot implementation), given that you document a folder structure that you will look for and pull.

If consumers go with Espresso, its' pretty simple to take a screenshot to any given folder with:

import androidx.test.runner.screenshot.*
Screenshot.capture(BasicScreenCaptureProcessor(File("/mypath")))

[Nilzor]

and the referenced allure branch actually does just that - pull a folder from the device and add the contents as artifacts of a particular test)

Does that include presentation in the HTML report as well? How far along is that branch?

[Malinskiy]

@Nilzor this branch is merged and is available in 0.6.0. The resulting report looks exactly as allure report looks like by default:

I agree that for the use-case when you just want to just pull some folder from the device (maybe containing screenshot taken with some lib) we should support it