TEAMMATES / teammates

This is the project website for the TEAMMATES feedback management tool for education
https://teammatesv4.appspot.com/
GNU General Public License v2.0
1.66k stars 3.31k forks source link

StudentHomePageUiTest: fix state leak #8524

Closed damithc closed 6 years ago

damithc commented 6 years ago

v6.3.0, live server StudentHomePageUiTest is failing due to a state leak from another test, most likely from StudentProfilePageUiTest. Specifically, the course below is showing up in StudentHomePageUiTest

Taken from StudentProfilePageUiTest.json

"SHomeUiT.CS2103": {
      "id": "SHomeUiT.CS2103",
      "name": "Software Engineering",
      "timeZone": "UTC"
    }

Location of the failure:

java.lang.AssertionError:  

    at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
    at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
    at teammates.test.driver.HtmlHelper.assertSameHtml(HtmlHelper.java:95)
    at teammates.test.driver.HtmlHelper.assertSameHtml(HtmlHelper.java:63)
    at teammates.test.pageobjects.AppPage.verifyHtmlPart(AppPage.java:865)
    at teammates.test.pageobjects.AppPage.verifyHtmlMainContent(AppPage.java:917)
    at teammates.test.cases.browsertests.StudentHomePageUiTest.testContentAndLogin(StudentHomePageUiTest.java:86)
    at teammates.test.cases.browsertests.StudentHomePageUiTest.allTests(StudentHomePageUiTest.java:49)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:659)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:845)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1153)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
    at org.testng.TestRunner.privateRun(TestRunner.java:771)
    at org.testng.TestRunner.run(TestRunner.java:621)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:357)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:352)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:310)
    at org.testng.SuiteRunner.run(SuiteRunner.java:259)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1199)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1124)
    at org.testng.TestNG.run(TestNG.java:1032)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

image

Probably the same student is used in both tests. May be delete courses of the student in concern during the setup phase?

On a related note, should not use the prefix SHomeUiT. in test data of StudentProfilePageUiTest.json. Instead, it should use something like SProfileUiT.

wkurniawan07 commented 6 years ago

I have some spare time this weekend to look into this. If someone else wants to do it before me, please be welcome.

tshradheya commented 6 years ago

Ideally the StudentProfilePageUiTest and StudentHomePageUiTest should not have any state leak because the latter belongs to sequential-ui-tests, right?

damithc commented 6 years ago

Ideally the StudentProfilePageUiTest and StudentHomePageUiTest should not have any state leak because the latter belongs to sequential-ui-tests, right?

@tshradheya Ideally, no two tests should have any state leaks between them. For sequential tests, it's enough if they clean up at the end so that no trace of it remains after the test is over. For parallel tests, whatever they do should not affect other tests even while the test is running.

In this case, you can start by running the two tests sequentially to see if one causes the other to fail.