TNG / JGiven

Behavior-Driven Development in plain Java
http://jgiven.org
Apache License 2.0
437 stars 98 forks source link

attachments start with attachment0 for each test class #144

Closed mgehlen closed 9 years ago

mgehlen commented 9 years ago

Hi,

I have now switched my mobile end-to-end tests completely to using BDD with JGiven and Appium as the actual test automation driver. I use testNG as a test runner and I separate my TestClasses alongside features.

My general structure is as follows:

public class MobileScenario extends ScenarioTest<GivenAppState, WhenActions, ThenAssertions>

MobileScenario uses testNG Dependency Injection to fetch the test configuration from maven ( i.e. test android or ios, which activity to start, which timeout to use, ...) and it defines some general tags for jGiven-reports.

public class LoginScenarioTest extends MobileScenario

This is an example for one of the actual test classes. It extends the MobileScenario mainly to get the Stages and the Tags. Then it just lists the scenarios to test and sometimes uses some DataProviders.

The interesting part in the stages might be the jGiven LifeCycle Annotations for ThenAssertions:

public class ThenAssertions extends Stage<ThenAssertions>
...
@AfterStage
    private void addScreenshotToReport() {

        Photographer photographer = new Photographer(appRunner.getDriver()) ;
        Attachment attachment = null;

        try {
            String timestamp = (new Timestamp(System.currentTimeMillis())).toString() ;
            attachment = Attachment.fromBinaryFile(photographer.takeScreenshot(), MediaType.PNG).withTitle("Screenshot_" + timestamp);
        } catch (IOException e) {
            e.printStackTrace();
        }

        currentStep.addAttachment(attachment) ;
    }

    @AfterScenario
    void resetAppRunner() throws IOException {

        appRunner.resetApp(this.port);
    }

So I reset the App after every scenario in order to give a clean and defined start for the next scenario so I don't need to worry if a scenario passed or failed in order for the next scenario to run properly.

Additionally I add a screenshot in AfterStage to my Scenario to show the last screen that appeared during a scenario to the report. This is the most interesting screen regadless of failure or success.

My problem now is: each feature class starts the attachament from new with attachment0, so if I ...

The first 3 screenshots from LoginScenarioTest are overwritten, hence the final reports shows the screenshots from RegistrationScenarioTest in the scenarios for LoginScenarioTest. Using withTitle did not solve the problem, since it is only added to the tooltip.

Can you make the withTitle the actual name of the attachement for screenshots? this way I can influence the filename all by myself during testing and get the correct screenshots in the report?

janschaefer commented 9 years ago

that is definitely a bug. Thanks for reporting!

janschaefer commented 9 years ago

Using withTitle for the filenames might not be that good, because I would have to escape/replace all special characters.

janschaefer commented 9 years ago

I could introduce a new method withFilename if you think that this would be useful in your case.

mgehlen commented 9 years ago

thank you for the quick response - if my understanding is correct withFilename ist a very good way for me to go.

ivan-masich commented 9 years ago

Hi,

Maybe it's better to use name of the test class (with package, for example com/example/TestClass/attachment0.txt) as sub-folders for grouping attachments, what do you think about this?

I have same problem, but in my case I'm using attachment to show all default application settings, so I have step "Given default settings" with "text" attachment to show 30+ settings and in this case usage of attachments is very nice way.

Generating for such attachment unique filename using method "withFilename" is not very nice as from my point of view because I have same step in 50+ tests.

ivan-masich commented 9 years ago

By the way, "withFilename" is also nice to have, in this way we can give user-friendly name for attachments (considering that it must be unique in scope of one test class).

janschaefer commented 9 years ago

Yes. By default a unique name for the attachments. Using subfolders would be an option. The withFilename would be optional.