search

openmainframeproject / cobol-check

A testing framework for Cobol applications
Apache License 2.0
76 stars 28 forks source link

Management of the Linkage Section and GETMAIN CICS #151

Open FALLAI-Denis opened 2 years ago

FALLAI-Denis commented 2 years ago

Hi,

I did not find in the documentation any information on the management of areas in Linkage Section and the processing of CICS GETMAIN / FREEMAIN instructions.

To test a classic subprogram which receives its parameters in Linkage Section, it is necessary to be able to feed these zones in a test, and it is therefore necessary that these zones point to an allocated memory. In the absence of a calling program, this must be the tool that allows the necessary memory to be allocated to the areas declared in the Linkage Section, (allocate memory, set address of area to the allocated memory). Note that there can also be zones declared in Linkage Section which are not parameters received, but which are used by the program to map other zones, (SET ADDRESS OF linkage-area TO ...).

To test a CICS program in batch, it is necessary to mock the CICS commands, and in particular those which manipulate the memory: GETMAIN, FREEMAIN, and probably other commands like READQ SET POINTER. In this case, the tool must also allow memory blocks to be allocated and Linkage Section zones to be associated with these memory blocks.

neopragma commented 2 years ago

Hello,

(1) The proof of concept for this project supported subprograms by having the user write a trivial main driver. Examples for the general case of subprograms are here https://github.com/neopragma/cobol-unit-test/blob/master/src/main/cobol/SUBPROG.CBL and here https://github.com/neopragma/cobol-unit-test/blob/master/src/main/cobol/SUBPROGD.CBL. Examples for CICS programs are here https://github.com/neopragma/cobol-unit-test/blob/master/src/main/cobol/CICSDEMO.CBL and here https://github.com/neopragma/cobol-unit-test/blob/master/src/main/cobol/CICSDRIV.CBL.

Please bear in mind we have not yet built that functionality in the new code base. I'm providing the references so you can see our approach.

(2) The team have not discussed the question of SET ADDRESS OF and similar. It is worth considering. Thanks for mentioning it.

(3) The approach for CICS will be similar to what was done in the proof of concept. In particular, we will not emulate CICS commands. We are not trying to write a CICS emulator. So we will not write an equivalent to EXEC CICS GETMAIN, for instance. The intent is for Cobol Check to run independently of CICS. So, all CICS commands will be stubbed by default, automatically. Users can (I should say, will be able to, because we haven't written this yet) create mocks for those commands per the needs of their test cases.

If you actually need a CICS emulator to run under Windows or Linux, you might take a look at this project: http://www.z390.org/.

As Cobol Check is meant for fine-grained unit testing at the level of individual paragraphs, we do not foresee a need to allocate and deallocate memory blocks. But I would need to consult my team mates about an approach to this. We might just move the Linkage Section declarations of those areas into Working Storage, as we currently do to stub batch I/O operations - we move record layout areas defined under an FD into Working Storage for purposes of running the tests.

See method processFileControlSource() in https://github.com/openmainframeproject/cobol-check/blob/main/src/main/java/com/neopragma/cobolcheck/Generator.java to see how we're currently doing that. Bear in mind that section of the code is presently being vigorously refactored to clean up the long if/else blocks. But you can see how we're handling memory areas that aren't allocated in the case of batch I/O. A similar approach could work for dynamically-allocated storage areas, too.

We have noticed a lot of people have a different concept of the scope of a unit test than we have. Our concept of a unit test is based on Michael Feathers' definition, summarized here: https://github.com/openmainframeproject/cobol-check/wiki/What-is-a-Unit-Test%3F

Sometimes the functionality people describe or request belongs at a higher level of abstraction than unit testing. The purpose of Cobol Check is to support unit testing only. We have a page on the wiki to describe alternatives to unit testing tools for testing at higher levels: https://github.com/openmainframeproject/cobol-check/wiki/Component-and-Integration-Testing.

Thanks very much for your interest!

FALLAI-Denis commented 2 years ago

Hi @davenicolette

Thanks for your answer.

(1) Your approach for dealing with subroutines and CICS programs (which receive a communication area like subroutines), is to request the writing of a driver program to allocate the parameters memory and to communicate them to the subroutine / CICS program. This adds workload to perform tests and increases the number of programs (almost 1 driver program per subroutine / CICS program). We believe that a better approach would be to allocate the necessary memory blocks and to associate the linkage zones in the initialization actions of the test suite, and directly execute the subroutine / CICS program without going through a driver program. This must be able to be done either by means of classic COBOL instructions like ALLOCATE linkage-area (available from IBM COBOL V6), or by commands specific to Cobol Check to erase the specificities of COBOL compilers (in this case Cobol Check must know the context execution target to adapt). Possibly see if the driver program could be "merged" with the program to be tested by transforming it into a "nested" program: a single source code, a single loadmodule.

(2) the mention of SET ADDRESS OF was just to indicate that the linkage zones are not necessarily zones for passing parameters, but could also be work zones of the program itself, (apply a description to a other area).

(3) for CICS programs, we did not imply that a CICS emulator was needed. Our idea is to test CICS programs in batch mode, but this requires mocking all CICS calls, and in particular those that return pointers to memory areas allocated by CICS, such as the READQ TS SET(pointer) instruction. In this case, the memory block is not allocated by the program to be tested and a memory allocation mechanism must therefore be available to mock the behavior of the CICS instruction.

We fully adhere to the principles outlined by Michael Feathers.

We are even considering being able to run z/OS mainframe COBOL programs outside the z/OS ecosystem (on the Developer Workstation, so for us under Windows). You just need to have a local compiler closest to the COBOL syntax supported by the IBM Enterprise Cobol compiler, (At one time IBM offered VisualAge COBOL for Windows, which was compatible with IBM Enterprise COBOL for z / OS V4, but not with version V6).

neopragma commented 2 years ago

Hi, I like the ideas you've suggested here. I'd like to keep the discussion going and engage the rest of the team to think about how we might implement this, and when to schedule it.

Another possibility is for you to join the team. That way, we could have this as a line of development alongside the other features we are working on. If you have an urgent need for this functionality, that might be the quickest way to get it. Something to consider.

It's interesting that you want to run COBOL on the developer workstation. The original idea behind Cobol Check was to enable unit testing outside the z/OS environment, to enable a TDD work flow for developers without the delay inherent in submitting jobs on-platform. Currently, Micro Focus has a compiler with IBM compatibility, and the open source project GNuCOBOL also has IBM compatibility. Cobol Check has mainly been tested using GnuCOBOL until now. We are explicitly multi-platform, and don't want to bake in any hard dependencies on Windows.

You mention running mainframe COBOL programs outside the z/OS ecosystem - for that, I still have to mention the www.z390.org project. It's out of scope for Cobol Check to enable running "any, random" mainframe programs off-platform. But it is in scope for that project. So far, they have pretty good emulation of the MVS instruction set, a pretty good implementation of high-level assembler, and the beginnings of support for COBOL, VSAM, and CICS.