search

IBM / zopeneditor-about

IBM Z Open Editor: File issues here!
https://ibm.github.io/zopeneditor-about
Apache License 2.0
50 stars 20 forks source link

Replacing Strings in Cobol Copy Books. #305

Closed RichyMueller closed 1 year ago

RichyMueller commented 1 year ago

Development environment used

Mainframe Environment

IBM z15 zOS 2.04

IMS DB2 MQSeries zOS Connect Cobol (4.2 and most 6.2) PL1 ASM Endevor

zosmf configured zowe configured

and al lot more
Our system consists of around 180000 Mainframe artifacts in 96 Groups of object types such as cb2, cpy, pli,inc, asm, mac, control cards and so on.

We are on the road to explore if we can move from Endevor with self written development environment (REXX) to IBM DBB and Gitlab for SCM and deployment.

Development environment should be vscode with zOpen Editor for the future.

Problem Description

We got Language Server Problems with nearly all of our copybooks.

Copys could not be properly used. Replacing String like " $TEXT$' and " 'AA-' "

could not resolved

Simple replacing

%VAR9 --------------------------------------------------------- %VAR9 AENDERUNGEN: %VAR9 JJJJMMTT ?WER %VAR9 ?WARUM *%VEND*** 05 'AA-'EPB-STATUS-AREA. 10 'AA-'STRUCTURE-INFO. 15 'AA-'STRUCTURE-ID PIC X(08). 88 'AA-'VALID-STRUCTURE-ID VALUE 'EPW0004C'. 15 'AA-'STRUCTURE-VERSION PIC X(05). 88 'AA-'CURRENT-STRUCTURE-VERSION VALUE '01.00'. 88 'AA-'VALID-STRUCTURE-VERSION VALUE '01.00'.

or

01 MY-KOM-AREA. COPY EPW0005C REPLACING =='AA-'== BY ==MY-== $PGM-ID$ BY ==VALUE 'EPU0015'==. or

01 LAST-LABEL-STACK GLOBAL. COPY EPW0021C REPLACING 'AA-' BY ==== $PGM-ID$ BY =='EPU0015'== $MAX$ BY ==50==.

Detailed steps for reproducing the problem:

  1. First step

use attached copys and programm

codeandcopy.zip

Observed behavior

See Pictures

EPU0015_CB2 EPW0005C_richyszosrepo

Expected behavior

-

FALLAI-Denis commented 1 year ago

Hi @RichyMueller,

I think ZOE is more respectful of the COBOL and COPY directive syntax than is the IBM compiler itself. What is accepted by the IBM compiler is rejected by ZOE...

image image $PGM-ID$ does not match any of the cases allowed for operand-1 because it is neither a literal nor an identifier, it is an illegal word for COBOL syntax. The $ character is not a separator, and is not allowed in an identifier name or a COBOL word.

This syntax works: image ==$PGM-ID$== corresponds to an authorized case for operand-1, that of the pseudo-text, and therefore is accepted by the COPY directive. But errors are detected at the level of the copybooks and affect the COPY directive in the program.

For copybooks, 'AA-' is not "well-formed", because the second apostrophe should be followed by a space, or a separator. The right syntax is the sequence apostrophe-space or apostrophe-separator : image This triggers parsing errors by COBOL Language Server in ZOE for copybooks, even though the copybook alone doesn't make sense before the transformation applied by the COPY directive.

I myself pointed out in various issues that ZOE, or more precisely the COBOL Language Server, should not have the same behavior as the COBOL compiler and should be much more tolerant of the COBOL syntax because the syntax cannot be validated only after resolution of the copybooks and also depends on conditional compilation and external elements to the source code as in the case of the use of the DEFINE(var,value) compilation option.

By using pseudo-texts for parameters and characters : as separators, all errors disappear except one: image It comes from the fact that copybook EPW0004C uses a reserved keyword when the AA- prefix is replaced by an empty string: image

Suggested implementation (but with impacts on the existing): codeandcopy_modified.zip

On the sidelines of your problem, for information, we are also in the process of setting up a modernization solution for our historic CI/CD mainframe chain with the withdrawal of two products:

Our replacement solution is based on:

We have defined a complete Git workflow to manage evolution and maintenance cycles, with parallel development management, (close to Gitflow with some adaptations to take into account the constraints of managing shared test environments).

We have a management of copybooks entirely based on the use of Git repositories (no more PDS for application copybooks). And this management is common to ZOE and DBB.

Our Development Workshop, based on VS Code and ZOE, offers increased productivity capabilities by automating part of the code (we had two different approaches: one based on code injection, automation of snippets and consistent management several injection points to implement the same functionality, and one based on code generation).

We would be happy to share with you our vision on this modernization of the mainframe CI/CD chain, perhaps through the discussion area of this site.

phaumer commented 1 year ago

Thanks so much for the analysis @FALLAI-Denis. We can confirm that we implemented our parser following the spec. We will discuss with the compiler team and investigate if we should relax this in our editors.

richydwp commented 1 year ago

Good Morning @FALLAI-Denis,

thank you for your detailed analysis. Sorry for the late answer. Other tasks had a higher priority.

Due to your various post and always excellent analysis of the reported issues and features we had a good idea what is going wrong and want to point out what issues will occur with existing mainframe systems when companys move to change the development and deployment environment away from TSO ISPF to a modern ci/cd pipeline.

We never get the developer community moved to a vscode or eclipse based editing environment when there is no really benefit to change. And such "simple" issues with copybooks / includes which all work since 30 years are a big hurdle to overcome ;-) Noone will pay to move from old, partly wrong, constructions to correctly named replacement tokens in copybooks if the whole system or big parts has to be retested. This is much to expensive.

Parts of our programming and styleguides came in place some 20 years agao. Some are from the last 10 years. The 'AA-' placeholder for Copybooks is one of the oldest one. That means in fact that we have around 17000 Copybooks with this construction. All modules with that copys are clean compilable.

We never get the order to change all the copys and as than neccesarry retest the whole mainframe based system. It is work So we need that the language server is more tolerant like the cobol compiler on the mainframe. Same is for §PGM-ID§ and some other language constructs.

Actually we had only a short look on pl1 modules with similar behavior of the language server. We had no look to our asm modules and macros. I only opened one asm module to look if it is ok.

We would really like to be in contact about yours and our mainframe modernization .

Actually we are on the road install DBB and all other requirements on the uss part to start a proof of concept in early january. I think we will come in a frequently discussion when we start our poc next year.

FALLAI-Denis commented 1 year ago

Hi,

I will be happy to share my experience on ZOE and on DBB with the community.

For DBB I think the site https://github.com/IBM/dbb-zappbuild will be more appropriate, (the topics concern more zAppbuild than IBM DBB itself).

More specifically on the CA Endevor migration to IBM DBB, I will ask the IBM team assisting us if there would be any interest in producing a WhitePaper. As there is an adherence also with ZOE (IBM User Build), this part could also be treated in this WhitePaper.

Perhaps disengaging from CA Endevor in favor of management under Git could be an opportunity to apply automatic transformations of your copybooks and sources at the time of migration, provided that there is standardization in your parameters. I don't think you will be able to coexist management under CA Endevor and management under Git for the same application set (subsystem). If you choose a solution similar to ours for the management of copybooks, (management entirely based on Git for builds by DBB without permanent PDS for storing copybooks), you could manage two sets of copybooks: those with historical standards, stored in PDS (Endevor outputs) and used by CA Endevor processors, and those in newer standards more compatible with ZOE, stored in Git Repositories and directly used by zAppbuild (and it's needed anyway to be able to trigger builds based on detection of modification of copybooks).

phaumer commented 1 year ago

@richydwp I agree. Please, feel free to post more issue like these that we can evaluate with the different teams in IBM to go beyond the language specification and support common practices with our editors as well.

phaumer commented 1 year ago

After talking to the compiler team we are now classifying this as a bug that we need to fix.

FALLAI-Denis commented 1 year ago

Hi,

For information: https://github.com/IBM/zopeneditor-about/discussions/184#discussioncomment-4725231

phaumer commented 1 year ago

Added support for $ and </> in v3.1.1.

RichyMueller commented 1 year ago

Works fine. Thank you