search

AbsaOSS / cobrix

A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
Apache License 2.0
138 stars 77 forks source link

Multi copybook parsing error #344

Open arykov opened 4 years ago

arykov commented 4 years ago

I am trying to parse a file that defined by a set of copybooks. So I set copybooks option to be populated by a set of comma separated file locations as code appears to suggest. I get an error "Copybook segments must redefine top segment". Not entirely clear what this means or how this can be addressed.

yruslan commented 3 years ago

If a file is defined by multiple copybooks that means that each copybook defines a schema for a segment. Copybooks are combined into a one copybook in which segment GROUPs redefine each other. For an example how to define segments that redefine each other, take a look at this section: https://github.com/AbsaOSS/cobrix#reading-hierarchical-data-sets

I'd recommend starting with combining multiple copybooks into one big multisegment one manually to get comfortable with the concept of loading multisegment files using Cobrix. Then, if it works for you, you can try defining segments in separate copybooks so that segments could be combined automatically.

You can see how copybook merging works here: https://github.com/AbsaOSS/cobrix/blob/master/cobol-parser/src/test/scala/za/co/absa/cobrix/cobol/parser/copybooks/MergeCopybooksSpec.scala

If you need help of combining copybooks, please send me an example copybooks that you have. You can rename/remove fields that are part of your IP, I just need general structure, such as level of fields.