pc2ccs / pc2v9

Version 9 of the PC^2 Programming Contest Control System
Eclipse Public License 2.0
46 stars 23 forks source link

Create JUnit(s) for testing the ShadowController class. #58

Open clevengr opened 4 years ago

clevengr commented 4 years ago

ShadowController is a non-GUI class which was created to provide overall management of shadowing operations. Because it was written during a time of intense development (in the ramp-up to NAC 2020), no JUnits were initially included. Several opportunities for creating JUnits for testing this class probably exist, and if viable these should be implemented.

Among the elements of ShadowController that are candidates for JUnit creation are testing the method getJudgementComparisonInfo(), which returns a Map from SubmissionIDs to ShadowJudgementInfo objects describing the current judgements assigned by the PC2 shadow system as well as the Remote CCS being shadowed.

However, it's not completely clear exactly how to create such a JUnit, since it needs to instantiate a ShadowController and start it shadowing -- but to do so the ShadowController must make a connection to a "remote CCS" to obtain judgements to be reported in the judgementComparisonInfo. But the JUnit can't rely on the availability of an external CCS running a contest at any particular time the JUnit is executed.

Perhaps the JUnit could use a Mock CCS Adapter, but there are at least two potential problems with this. First, it's not known at this writing (12 march 2020) whether the Mock CCS Adapter class is even fully functioning (it wasn't, as of the last time I worked on it, when I switched to using a Real CCS Adapter just prior to going to NAC in February 2020). Second, it's not known whether the "Paced Replay" function of the Mock adapter is working, or whether the timing of the "Paced" version is reasonable to be used in conjunction with a JUnit.

In addition, it's not clear what defines "success" for a JUnit like this; it could call getJudgementComparisonInfo() multiple times, but then what? Clearly, some non-trivial design work needs to be done to figure out what constitutes an appropriate JUnit for this class.

lane55 commented 4 years ago

Excellent comment. The discussion and questions are: What is a good JUnit scope ? What considerations should happen when writing code to make JUnit tests easier to write? What other testing, non-JUnit, should be in the system? In order to do JUnits and to communicate to any contributes unit testing should have at a minimum some guidelines to answer the questions from above.

lane55 commented 1 year ago

At a minumum there are fields/structures that should be moved out of ShadowController into a new model class. A good guide to creating unit tests is small fast and test one method test case.

lane55 commented 1 year ago

Answers to questions posed above.

| ShadowController is a non-GUI class which was created to provide overall management of shadowing operations. Because it was written during a time of intense development (in the ramp-up to NAC 2020), no JUnits were initially included. Several opportunities for creating JUnits for testing this class probably exist, and if viable these should be implemented.

JUnit unit testing is not typically used to test UI activities, it is to test the controller and model methods.

| Among the elements of ShadowController that are candidates for JUnit creation are testing the method getJudgementComparisonInfo(), which returns a Map from SubmissionIDs to ShadowJudgementInfo objects describing the current judgements assigned by the PC2 shadow system as well as the Remote CCS being shadowed.

Yes.

| However, it's not completely clear exactly how to create such a JUnit, since it needs to instantiate a ShadowController and start it shadowing -- but to do so the ShadowController must make a connection to a "remote CCS" to obtain judgements to be reported in the judgementComparisonInfo. But the JUnit can't rely on the availability of an external CCS running a contest at any particular time the JUnit is executed.

As follows - use the MockContestAPIAdapter.

| Perhaps the JUnit could use a Mock CCS Adapter, but there are at least two potential problems with this. First, it's not known at this writing (12 march 2020) whether the Mock CCS Adapter class is even fully functioning (it wasn't, as of the last time I worked on it, when I switched to using a Real CCS Adapter just prior to going to NAC in February 2020). Second, it's not known whether the "Paced Replay" function of the Mock adapter is working, or whether the timing of the "Paced" version is reasonable to be used in conjunction with a JUnit.

If there are bugs/improvements to aid in unit testing, those should be fixed/done.

| In addition, it's not clear what defines "success" for a JUnit like this; it could call getJudgementComparisonInfo() multiple times, but then what? Clearly, some non-trivial design work needs to be done to figure out what constitutes an appropriate JUnit for this class.

Success is based on the method tested. As indicated there are design changes needed, esp to create a shadow model to complement the shadow controller.