COBOL COPYBOOK Improperly reporting TOKEN errors

[gecolbert]

Development environment used

• Z Open Editor version:
• Editor Platform
  • [X] Visual Studio Code
  • [ ] Red Hat CodeReady Workspaces
  • [ ] Eclipse Che
  • [ ] Standalone Theia
• Editor Platform Version:
• Operating System (on which VS Code runs such as Windows 10 2004, otherwise name and version of platform such as OpenShift v4.3):Windows 10 21H1
• Java Version (when using VS Code or Theia, execute java -version and paste the details here):
• [ ] Related to RSE API?
  • RSE API Plugin version:
  • Zowe CLI version:
  • Node.js version:
• Logs attached (see here how to get them): yes/no

Problem Description

Detailed steps for reproducing the problem:

1. First step Edit a Cobol COPYBOOK with multiple tokens: 000001 01 :PREF:-LOC-ADDR-TABLE-REC. 000002 :LB: :PREF:-LOCALITY-TABLE. 000003 :LC: :PREF:-LOCALITY-TRLR 000004 OCCURS 0 TO 35 TIMES 000005 DEPENDING ON :PREF:-NUM-OF-TRAILERS 000006 INDEXED BY :PREF:-TRLR-INDX. 000007 :LD: :PREF:-LOCALITY-FROM-DATE 000008 PIC S9(07) COMP-3. 000009 :LD: :PREF:-LOCALITY-THRU-DATE 000010 PIC S9(07) COMP-3. 000011 :LD: :PREF:-LOCALITY-CDE 000012 OCCURS 102 TIMES 000013 ASCENDING KEY IS :PREF:-LOCALITY-CDE 000014 INDEXED BY :PFIDX:LOC-INDX 000015 PIC X(02). 000016 :LD: FILLER PIC X(10). 000017

zOpen Editor reports multiple TOKEN ERRORS

Expected behavior

Should be accepted as valid COPYBOOK structure

[FALLAI-Denis]

Hi @gecolbert

You touch on one of the problems I expressed in discussion #184 The language server is not a language compiler.

That said, if the description of your COPYBOOK is perfectly legal in terms of form, I think there is a design problem in substance: the level numbers should not be symbolic parameters, and I doubt that it there really is a need for them to be. You could just as easily put real values on your number levels and do a replacement when calling COPYBOOK (the IBM documentation on the compiler gives examples of this use case). As far as we are concerned, our COPYBOOKs of data descriptions do not contain any symbolic parameters and are "totally COBOL", which does not prevent us from making replacements during COPY. This has another advantage: our COPYBOOKs can be used in our data manipulation tool which can associate a (actually) COBOL COPYBOOK with a file and interpret its content by means of the COPYBOOK.

Example:

Despite this, unless I am mistaken, the language server detects a false error because it misinterprets REPLACING ...

[FALLAI-Denis]

To solve the initial problem, the language server, but also the textmate grammar, would have to accept at the place where a level number is expected, a text answering the following pattern:

/0[1-9]|[1-4][0-9]|66|77|88|(?P<sep>[\:])(?:\w|\-)+(?P=sep)/

From https://regex101.com/

1st Alternative 0[1-9]

• 0 matches the character 0 with index 48d (30h or 60o) literally (case insensitive)
• Match a single character present in the list below [1-9]
• 1-9 matches a single character in the range between 1 (index 49) and 9 (index 57) (case insensitive)

2nd Alternative [1-4][0-9]

• Match a single character present in the list below [1-4]
• 1-4 matches a single character in the range between 1 (index 49) and 4 (index 52) (case insensitive)
• Match a single character present in the list below [0-9]
• 0-9 matches a single character in the range between 0 (index 48) and 9 (index 57) (case insensitive)

3rd Alternative 66

• 66 matches the characters 66 literally (case insensitive)

4th Alternative 77

• 77 matches the characters 77 literally (case insensitive)

5th Alternative 88

• 88 matches the characters 88 literally (case insensitive)

6th Alternative (?P<sep>[\:])(?:\w|\-)+(?P=sep)

• Named Capture Group sep (?P<sep>[\:])
• Match a single character present in the list below [\:]
• \: matches the character : with index 58d (3Ah or 72o) literally (case insensitive)
• Non-capturing group (?:\w|\-)+
• + matches the previous token between one and unlimited times, as many times as possible, giving back as needed (greedy)
• 1st Alternative \w
• \w matches any word character (equivalent to [a-zA-Z0-9_])
• 2nd Alternative \-
• \- matches the character - with index 45d (2Dh or 55o) literally (case insensitive)
• (?P=sep) matches the same text as most recently matched by the capturing group named sep

[FALLAI-Denis]

About last (or second problem) concerning "replacing" on "ascending key is", this works: put "PREF-LOCALITY-CDE" as a subordonate entry of the OCCURS.

I think where is a problem un Language Server that don't accept "ASCENDING/DESCENDINK KEY" on the subject of the OCCURS.

Perhaps need to open a new issue for this second problem ?

[gecolbert]

The structure of this copybook is perfectly legal (for Enterprise Cobol) in terms of form.

While I fully agree that the use of tokens in this structure is the result of an overly exuberant programmer and are completely un-necessary, the obstacles to fixing it in my system make the implementation of z Open Editor nearly impossible.

This copybook was introduced into my production system in 1992. It is a core part of the processing for billions of dollars of transactions every month. Just changing the :LB: tag to an 05 will require a three month long project of obtaining change approval, scheduling of the distribution of the change to multiple external agencies, and complete end-to-end testing of our entire system. All of this not to correct a problem or enhance any system functionality.

The arduous effort of change management to critical systems makes it necessary for any new tools to be capable of properly emulating the Enterprise Cobol compiler's behavior. I need an editor that fully implements the language without having to make system changes to accommodate the editor.

[FALLAI-Denis]

Hi @gecolbert

To avoid problems linked to the language server, you can pause it: "F1" + "pause"

Cf. #154 Cobol Language Server - "Full red program" after LS analysis.