aws-samples / mainframe-data-utilities

Python scripts to convert and unpack mainframe EBCDIC data on the cloud or any ASCII environment.
Apache License 2.0
36 stars 12 forks source link

When will OCCURS DEPENDING ON be supported #10

Closed elhadi412 closed 1 year ago

elhadi412 commented 1 year ago

Hi - I see that the following 2 items are in the backlog for the copybook parser:

When do you expect these 2 items will be supported?

luisdant commented 1 year ago

Hello.

Actually, there is already some support for OCCURS DEPENDING ON.

For 'Data item REDEFINES' there is no date yet. Do you have an idea how to do this? contributions are welcome.

It's important to mention that GROUP ITEM REDEFINES are supported (edited).

Regards,

Luis

elhadi412 commented 1 year ago

Unfortunately no, I do not have an idea on how to handle REDEFINES data item. Although group item REDEFINES proved useful, we do have a use case where we have data item REDEFINES. Regarding OCCURS DEPENDING ON, can you speak more to the some support part?

luisdant commented 1 year ago

Is the data item redefines in an huge copybook? Can you share a small sample?

The DEPENDING ON works for files which contain this clause at the trailing columns. At Local Single Layout VB you will find an example of a file conversion where the copybook is in this situation.

The extraction script ignores the DEPENDING ON and creates empty data fields from empty content.

00|01|001|000000001|NAME NUMBE000000001||||||||||||||||||
00|02|002|000000001|NAME NUMBE000000001|000000002|NAME NUMBE000000002||||||||||||||||
00|03|003|000000001|NAME NUMBE000000001|000000002|NAME NUMBE000000002|000000003|NAME NUMBE000000003||||||||||||||
elhadi412 commented 1 year ago

Hey - sorry for the late response. I am not able to share a sample, it is proprietary data unfortunately. We were able to find a work around. There is a tool called Cobrix that implements spark and converts these copybooks into data streams (dataframes in spark terms).

Thanks