Feature 288 enable cobol check to issue warnings/errors when a call statement is not mocked

[AkashKumar7902]

Issue: #288

[Rune-Christensen]

Hi @AkashKumar7902 I have again failed in describing what is needed for solving this issue. I am sorry for that.

Your code is great, but it isn't necessary to keep track of which tests use which mocks, to solve this issue.

If the configuration is set so, that there should be errors when a call is not mocked, you should add code to every commented call. No matter if the call is used or not.

After every commented call, there should be a bit of code, that is executed instead of the commented call. Like I described in your last pull request. https://github.com/openmainframeproject/cobol-check/pull/316#issuecomment-1702491435 This code should add 1 to a counter, and display that there is an error. The counter should not be reset during the run, and at the end of the run, the code should check that the counter is still zero. If it is not zero, it should state that there are unmocked calls, and the unit test should fail.

I hope this clears it up. Rune

[AkashKumar7902]

Hi @Rune-Christensen sorry but I think I am still unable to understand the proposed changes.

I would like to express my thoughts after going through your comment in my last pull request, so that you can confirm whether we were on the same page.

lets take an example,

suppose there is a unit test with testcase name "testcase" and testsuite name "testsuite" and in the code block for this test, there is a perform statement "PERFORM 100-PARA" ,

TestSuite "testsuite"
TestCase "testcase"
PERFORM 100-PARA"

the source code which is under the test has the following code inside 100-PARA (it is the same paragraph which is referenced in perfom statement in our above testcase)

100-PARA.
     CALL 'PROG' 

The code under "testcase" should contain a mock for all the calls inside 100-PARA, but since it is missing the mock for CALL 'PROG' the test case should fail. I believe this is the intended behavior you want.

In the below code you mentioned in my last PR,

100-Demo section.
* Call 'dummy'
Add 1 to ==UT==NUMBER-FAILED
display "Call not mocked in testcase "testcasename" in testsuite "testsuitename" 
display "All used calls should be mocked, to ensure the unit test has control over input data"

In the event of analyzing the above code, the below was my thought process:

The UT-NUMBER-FAILED is keeping track of the number that is going to be prefixed with FAILED which gets showed at the end of the execution of ./cobolcheck -p {programname} in the terminal :

  8 TEST CASES WERE EXECUTED
  5 PASSED
  3 FAILED
=================================================

Increasing or decreasing this number anywhere else other than CCHECKPARAGRAPHSPD.CPY copybook will result in the test summary to be statistically wrong. (5 passed + 4 failed = 9 test cases) :

  8 TEST CASES WERE EXECUTED
  5 PASSED
  4 FAILED
=================================================

To fail the test case, we have CCHECKPARAGRAPHSPD.CPY copybook, in this copybook whether the unit test fails or not is governed by ==UT==COMPARE-PASSED.

so i changed CCHECKPARAGRAPHSPD.CPY to fail the test if the variable ==UT==UNMOCK-FAILED is set to 1 (true), this is reintialized before each testcase so that the same value of ==UT==UNMOCK-FAILED does not propagate to further testcase.

Please can you check in which part I was wrong or If i interpreted it completely wrong :( Sorry If i assumed and interpolated a lot :(

Akash Kumar

[Rune-Christensen]

Hi @AkashKumar7902

Great what you are doing, lets hash this out completely, before continueing :-)

Regarding the output, I was thinking that the ==UT==NUMBER-FAILED counter would be used for a new line in the output:

  8 TEST CASES WERE EXECUTED
  4 PASSED
  4 FAILED
  3 CALLS NOT MOCKED
=================================================

In that way, you do not have to reset the counter after each test, you do not have to check it in each verify and expect statement, and you can just verify that the counter is zero. If it is zero, you can omit the extra line. And the user can see that there are three unmocked calls, and look for the displays from the unmocked calls in the output.

Regarding where to place the extra code. You do not have to keep track of each call compared to each testcase. Just add the extra code at every call. Making sure that the user can identify which call is not mocked. If a call is not part of the code executed in a testcase, the extra code is not run, and everything is fine, because the user does not have to mock a call that is not being used. And if a call is used in the testcase, then the code is present, and will notify the user of the issue.

I hope this helps clearing up the issue. Regards Rune

[AkashKumar7902]

@Rune-Christensen Thanks for the clear description. I believe i am now able to understand fully what we are trying to accomplish. I have created a new PR with the changes: #318

Regards Akash Kumar