Subdescriptors

[awilkins]

Fixes #506

Not sure about whether I need to add the subdescriptor fields to another collection ; superdescriptors reference existing fields, subdescriptors create new fields that are views on their parent (?).

You're not allowed to mix them, I think, which is how we are deciding whether it's super or sub.

[MarkusAmshove]

Side note: You can format the code with gradlew spotlessApply (documentation) if the check CI / Check formatting (pull_request) fails. That way your code will automatically get formatted to keep the diffs smaller :)

[MarkusAmshove]

If I understand the documentation correctly, then the following (previously unknown to me) rules apply:

• Subdescriptors have to be defined after fields
• Superdescriptors have to be defined after subdesriptors or fields

With that in mind, I would change the parsing like this:

• Find S and start parsing a superdescriptor (the S field itself should be parsed by then)
• Parse the first child of the superdescriptor
• If the child can be referenced (because fields have to be declared beforehand), then proceed normaly
• Otherwise mark the S field as subdescriptor instead of superdescriptor
• add childs as normal descriptors with the datatype of the S field and the range given

So having the following DDM:

DB: 000 FILE: 100  - COMPLETE-DDM                      DEFAULT SEQUENCE: 
TYPE: ADABAS
T L DB Name                              F Leng  S D Remark
- - -- --------------------------------  - ----  - - ------------------------
  1 AC ALPHA-FIELD                       A    8  N
  1 AB ANOTHER-NUMBER                    N   12  N
  1 AG A-SUBDESCRIPTOR                   A   20  N S
*      -------- SOURCE FIELD(S) -------
*      A-SUB-FIELD   (1-8)
*      SUB-FIELD-TWO (1-12)

The DDM should contain these fields:

• ALPHA-FIELD (A8)
• ANOTHER-NUMBER (N12)

and these descriptors:

• A-SUBDESCRIPTOR (A20) type SUBDESCRIPTOR
• A-SUB-FIELD (A8) type DESCRIPTOR
• SUB-FIELD-TWO (A20) type DESCRIPTOR

I'm unsure about the descriptor type of A-SUBDESCRIPTOR, because the documentation states that it might be a descriptor itself, but we don't know how predict generates those sources.

---

Your changes don't raise any issues on our 6M LoC code base, so I don't have a problem with merging your changes, I'd do a few renames though.

We have a few choices now, you can decide :-)

• Pursuit your changes to make it work for you
• I can take over the implementation of the issue and do a semi-big rewrite of the logic

The DDM parser will be rewritten in the future to enable the output to be used in the language server, but I've not planned when I come around to do it.

[awilkins]

Apologies for the intermittent communication : I work for this client Tue-Thu so there can sadly be these four day gaps ... my goal here is "eliminate all the ERROR level problems for code that clearly compiles and runs on the mainframe" and the codebase is ... on a similar order of magnitude in size (not as large as yours though), and several decades old, so hopefully we both get something out of collaborating :-)

I'd love to get to the point where I can put the sonar job in a CI pipeline and benefit from all that lovely linting, but even being able to use VSCode fluently is better than NaturalONE. Honestly, I started to write a tree-sitter parser for Natural and rapidly realized it would drive me mad before I got useful results.

I think the semi-big rewrite sounds better - you know your code better, you will know better how to refactor it. I'm at the stage where all I can do is copy how you've done something already.

I have more red squigglies to submit issues / PRs for :-)

We don't know how predict generates those sources.

If there are any experiments I can do for you I can try to get them done.

[sonarcloud[bot]]

Quality Gate failed

Failed conditions
2 Security Hotspots

See analysis details on SonarCloud

[MarkusAmshove]

No problem, I'll close this PR and see what I can do. I'll @ you in the new one so you can get an idea what I was talking about :)

In our code base we have around 3000 parser diagnostics left. Some of them are still missing parts in the parser, others are actual runtime errors (like PERFORM to non existing subroutines, CALLNATs etc.). For us this is such a usable state, that almost all our developers (aside from 1-2) have switched to the Language Server for their daily work :)

If you want to summarize those issues better, you can run the linter CLI in your project root like this:

java -jar natlint.jar --sink=CSV -xlint

-xlint skips the linting, so only parser errors will come up --sink=CSV will write all diagnostics into a diagnostics.csv file

If you ever want to have a chat, feel free to send me an email to the address associated with my commits, I can send you my MS Teams contact then.

Claes is also working on mainframe support, they have a huge Natural project too :-)