Issue with Skip tests.

[pavankumar9]

Even if the log status is set to skip. The tests are shown as "Pass". Is there any solution for this?

[pavankumar9]

Seems like this part:

private void trackLastRunStatus(LogStatus logStatus) { if (runStatus == LogStatus.UNKNOWN) { if (logStatus == LogStatus.INFO) { runStatus = LogStatus.PASS; } else { runStatus = logStatus; }

        return;
    }

    if (runStatus == LogStatus.FATAL) return;

    if (logStatus == LogStatus.FATAL) {
        runStatus = logStatus;
        return;
    }

    if (runStatus == LogStatus.FAIL) return;

    if (logStatus == LogStatus.FAIL) {
        runStatus = logStatus;
        return;
    }

    if (runStatus == LogStatus.ERROR) return;

    if (logStatus == LogStatus.ERROR) {
        runStatus = logStatus;
        return;
    }

    if (runStatus == LogStatus.WARNING) return;

    if (logStatus == LogStatus.WARNING) {
        runStatus = logStatus;
        return;
    }

    if (runStatus == LogStatus.PASS || runStatus == LogStatus.INFO) {
        runStatus = LogStatus.PASS;
        return;
    }

    runStatus = LogStatus.SKIP;        
}

I am having an info statement on Suite Start.. so it doesn't recognise it as skip. If i remove it i ll treat as skipped test.

[testphreak]

I see the same behavior. A skipped test is reported as a pass. On the other hand skipped tests due to a configuration failure get reported correctly as skip.

public class SkipTestClass {

    @Test
    public void skippedTestMethod() {
        throw new SkipException("Test Skipped");
    }
}

[anshooarora]

This is by design because if any step other than SKIP is logged means the test wasn't really SKIPPED.

[testphreak]

Not sure I followed you Anshoo. For a test method that throws a SkipException, both TestNG and Maven report the test as Skipped.

Tests run: 5, Failures: 2, Errors: 0, Skipped: 1

But the same suite in ExtentReports shows 3 tests passed including the skipped one and 2 failures.

[anshooarora]

So the priority of a SKIP test should be above INFO/PASS? Actually it should be above INFO even as per the hierarchy mentioned in docs.

[testphreak]

Sure, perhaps I am not following you, but I thought we should simply report the test as Skip. If you feel strongly about reporting a skipped test as a Pass that's fine. Maybe it will hit me after some time what you're implying :smiley:. Could you also point to the docs, so I can learn more about the hierarchy?

[anshooarora]

Hierarchy here. Changing this behavior is fine especially as i would like it to be inline with existing test frameworks.

Not all status types are available in either frameworks but how does this sound:

Fatal Fail Error Warning Skip Pass Info Unknown

[pavankumar9]

Basically when we are following data driven test we cannot throw Skip Exception as it aborts the whole test. Instead we just indicate the corresponding test data as skipped. It would be good if we mark the status as "Skipped" if any of the step is logged with status as skipped. It gives the whole representation of test execution.

[testphreak]

Anshoo, yes, It's true not all statuses in ExtentReports are in TestNG, Maven or JUnit, but I think ExtentReports should match the behavior for the 3 major ones, i.e. Pass, Fail and Skip.

Thanks for linking the doc. Your updated list above looks correct. Essentially you're flipping Pass and Skip and ranking Skip higher. So the test with this updated hierarchy would report as Skip.

But what is the behavior then for passed tests? Will they start reporting as Skip since Skip is above Pass? Hmm... that doesn't make sense, or maybe I need to better understand the hierarchy concept.

[testphreak]

Oops I stand corrected. Your updated hierarchy would report passed tests as a pass as long as the steps I log use Info, Pass or Unknown for logging with the tests. Looks good to me!

[anshooarora]

The new hierarchy is included in beta-5, available here.

[testphreak]

Great! Thanks Anshoo!

[email2vimalraj]

The FAQ has to be updated for this.

[JobvandenBerg]

Isn't it possible to tweak the LogStatus hierarchy, with ofcourse a default setting? When I SKIP steps in my testcases, I'd like it to say that it passed the test, but skipped one step. When I SKIP an entire test(or all individual steps), the testcase can be set as SKIP, perhaps?

[email2vimalraj]

@JobvandenBerg : I'll call that as a special case. Universally, even if one step is skipped, the test should be marked as skipped.

[Hanumant1987]

Hello Anshoo,

I was having the same issue I was using version extentreports-2.03. After following your fix. downloaded extentreports-java-2.41.1 in my eclipse. But all stopped working. Please let me know if I am doing something wrong.

My BeforeTest function looks like this where I want to use Skip functionality as I need to skip test cases which are marked as skip in my input excel.

Or in any way can I use extentreports-2.03. only and apply skip in my test report.

public void beforeTestDataToRun(Read_XLS FilePath, int DataSet,String TestDataToRun[], String TestName, String TestCaseName , ExtentTest test , ExtentReports extent) throws IOException { if(!TestDataToRun[DataSet].equalsIgnoreCase("Y")){

          Add_Log.info("TestCase-" + TestCaseName + "_TestName "+  TestName+" : DataToRun = N for data set line "+(DataSet+1)+" So skipping Its execution.");
          Testskip=true;
         // test.log(LogStatus.PASS, "Test Passed as " + "ActualResult Value " +" ' "+  ActualResult + " ' "+ " And ExpectedResult Value "  +" ' "+ ExpectedResult + " ' "+ " Matched" );
        test.log(LogStatus.SKIP ,  TestName+" : CaseToRun = N for So Skipping Execution.");

          Add_Log.info("TestCase-" + TestCaseName + "_TestName "+  TestName + " : Reporting test data set line "+(DataSet+1)+" as SKIP In excel.");
          SuiteUtility.WriteResultUtility(FilePath, TestCaseName, "Pass/Fail/Skip", DataSet+1, "SKIP");

          throw new SkipException("DataToRun for row number "+DataSet+" Is No Or Blank. So Skipping Its Execution.");
      }

}

[anshooarora]

@Hanumant1987 - not sure exactly what is not working as per your post, but see here:

http://extentreports.relevantcodes.com/java/#testng-example

[Hanumant1987]

Hello Anshoo,

Thanks for your quick reply. Actually code is working fine but as mentioned by you that Skip wont make test status Skip but makes Pass only, it only mention test step skip. So you have fixed it as I saw in discussion. But when I tried using version 3, its all new code I have to do, so I was just wondering is there any way I can make my test Skip by using test.log(LogStatus.SKIP , "") only using same only version extentreports-2.03.

[anshooarora]

What version are you using now? 2.4x or 3?

Both would mark test as skipped if all other status are info and pass.

PS. please open a new ticket to discuss this.

[Hanumant1987]

Hello Anshoo, I was using version-2.03 . Now I am using 3 and changed my code accordingly, and it's working fine for me thanks.

But with version 3 I am not able to see screen shot in report. Screen shot is attached but not opening.

Can you help. Please find my code below.

    public static String CaptureScreen(WebDriver driver, String TestName)

{ String destDir = "C:\OrionAutomation\TestReportScreenShot"; DateFormat dateFormat1 = new SimpleDateFormat("dd-MMM-yyyy-hh-mm a"); String destDir1 = destDir + "/" + dateFormat1.format(new Date()); new File(destDir1).mkdirs(); String Path = destDir1 + "/" + TestName + "" + dateFormat1.format(new Date()); System.out.println(Path);

TakesScreenshot oScn = (TakesScreenshot) driver; File oScnShot = oScn.getScreenshotAs(OutputType.FILE); File oDest = new File(Path+".jpg"); try { FileUtils.copyFile(oScnShot, oDest); } catch (IOException e) {System.out.println(e.getMessage());} return Path+".jpg"; }

// Function calling in test case

test.log(Status.INFO, "ScreenShot below: " + test.addScreenCaptureFromPath(ExtentManager.CaptureScreen(driver,TestName)));

Please find below screen shot of my generated report. This screen shot is saved in desired location but not opening in the report.

[Vinayakbansalgithub]

Hello Anshoo,

I am not able to change the status from skip to pass

Design is we have few steps in between which has been skipped but after those steps we have some steps marked as passed so we want overall test case to be passed but what i am getting is that if we have any of the step which we mark as fail then the status is not changing.

public static void Skip(String eSkipMessage) { if (Constant.HTMLREPORTING == true) { System.out.println(extent.getRunStatus()); extent.log( LogStatus.SKIP, ExcelUtils.getCellData(TestCaseRunner.testScriptIndex, Constant.TestStepID), ExcelUtils.getCellData(TestCaseRunner.testScriptIndex, Constant.TeststepDescription) + eSkipMessage);

        System.out.println(extent.getRunStatus());

    }
}

public static void pass(String ePassMessage) {
    if (Constant.HTMLREPORTING == true) {

        if (Constant.ExtReportloglevel == 3) {
            String location = SupportLib.createScreenshot();
            extent.log(
                    LogStatus.PASS,
                    ExcelUtils.getCellData(TestCaseRunner.testScriptIndex,
                            Constant.TestStepID),
                    ExcelUtils.getCellData(TestCaseRunner.testScriptIndex,
                            Constant.TeststepDescription)
                            + ePassMessage
                            + extent.addScreenCapture(location));
        } else {

            System.out.println("interested"+extent.getRunStatus());
            extent.log(
                    LogStatus.PASS,
                    ExcelUtils.getCellData(TestCaseRunner.testScriptIndex,
                            Constant.TestStepID),
                    ExcelUtils.getCellData(TestCaseRunner.testScriptIndex,
                            Constant.TeststepDescription) + ePassMessage);

            System.out.println(extent.getRunStatus());
        }

    }

}

First i am running skip then pass