search

legsem / legstar.avro

A COBOL to Apache Avro translator
GNU Affero General Public License v3.0
9 stars 8 forks source link

Redefines clause #5

Closed alonsnir closed 9 years ago

alonsnir commented 9 years ago

i'm trying to generate schema from copybook with Redefines clause and getting the following exception:

[java] 516 [main] INFO com.legstar.avro.generator.Cob2AvroGenerator - COBOL to Avro generation for: /Users/alonsnir/work/legstar.avro.distrib-0.4.3-SNAPSHOT/samples/cobol/book-redefine.cob [java] 720 [main] ERROR com.legstar.avro.generator.Cob2AvroGenerator - COBOL to Avro generation failed for: book-redefine [java] com.legstar.avro.translator.Xsd2AvroTranslatorException: XML schema does not contain a root element [java] at com.legstar.avro.translator.Xsd2AvroTranslator.translate(Xsd2AvroTranslator.java:180) [java] at com.legstar.avro.translator.Xsd2AvroTranslator.translate(Xsd2AvroTranslator.java:137) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generateAvroSchema(Cob2AvroGenerator.java:238) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generate(Cob2AvroGenerator.java:143) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generate(Cob2AvroGenerator.java:95) [java] at com.legstar.avro.generator.Cob2AvroGeneratorMain.generate(Cob2AvroGeneratorMain.java:42) [java] at com.legstar.base.generator.AbstractCob2JavaGeneratorMain.execute(AbstractCob2JavaGeneratorMain.java:106) [java] at com.legstar.avro.generator.Cob2AvroGeneratorMain.main(Cob2AvroGeneratorMain.java:28)

here is the copybook 01 WS-OLD PIC X(10). 01 WS-NEW1 REDEFINES WS-OLD PIC 9(8). 01 WS-NEW2 REDEFINES WS-OLD PIC A(10).

does it support redefines? could someone suggest how this cold be implemented if it doesn't?

Thanks

fmoussallam commented 9 years ago

I believe the issue here is that your copybook does not define a structure. There is no hierarchy.

Maybe you should try:

        01 FILLER.
            05 WS-OLD PIC X(10).
            05 WS-NEW1 REDEFINES WS-OLD PIC 9(8).
            05 WS-NEW2 REDEFINES WS-OLD PIC A(10).
alonsnir commented 9 years ago

hi fmoussallam, thank you for your reply.

i have tried the copybook as you suggested, but it fails with same error. any thoughts?

legsem commented 9 years ago

I doubt that it fails with the same error. Can you double check?

alonsnir commented 9 years ago

for the quicker evaluation i'm using the CustdatReader sample, so i'm trying to generate schema out of copybook using the sample. let me just describe what i did and recheck the error: have built from sources, copied my copybook into the legstar.avro.distrib-0.4.3-SNAPSHOT/samples/cobol/book-redefine.cob

$ cat ./samples/cobol/book-redefine.cob 01 FILLER. 05 WS-OLD PIC X(10). 05 WS-NEW1 REDEFINES WS-OLD PIC 9(8). 05 WS-NEW2 REDEFINES WS-OLD PIC A(10).

And now the ant build

Alons-MacBook-Pro:legstar.avro.distrib-0.4.3-SNAPSHOT alonsnir$ ant Buildfile: /Users/alonsnir/work/legstar.avro.distrib-0.4.3-SNAPSHOT/build.xml init: generate: [java] 0 [main] INFO com.legstar.base.generator.AbstractCob2JavaGeneratorMain - Generation started [java] 23 [main] INFO com.legstar.avro.generator.Cob2AvroGeneratorMain - Processing COBOL file /Users/alonsnir/work/legstar.avro.distrib-0.4.3-SNAPSHOT/samples/cobol/book-redefine.cob [java] 359 [main] INFO com.legstar.avro.generator.Cob2AvroGenerator - COBOL to Avro generation for: /Users/alonsnir/work/legstar.avro.distrib-0.4.3-SNAPSHOT/samples/cobol/book-redefine.cob [java] 514 [main] ERROR com.legstar.avro.generator.Cob2AvroGenerator - COBOL to Avro generation failed for: book-redefine [java] com.legstar.avro.translator.Xsd2AvroTranslatorException: XML schema does not contain a root element [java] at com.legstar.avro.translator.Xsd2AvroTranslator.translate(Xsd2AvroTranslator.java:180) [java] at com.legstar.avro.translator.Xsd2AvroTranslator.translate(Xsd2AvroTranslator.java:137) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generateAvroSchema(Cob2AvroGenerator.java:238) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generate(Cob2AvroGenerator.java:143) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generate(Cob2AvroGenerator.java:95) [java] at com.legstar.avro.generator.Cob2AvroGeneratorMain.generate(Cob2AvroGeneratorMain.java:42) [java] at com.legstar.base.generator.AbstractCob2JavaGeneratorMain.execute(AbstractCob2JavaGeneratorMain.java:106) [java] at com.legstar.avro.generator.Cob2AvroGeneratorMain.main(Cob2AvroGeneratorMain.java:28) [java] 516 [main] INFO com.legstar.base.generator.AbstractCob2JavaGeneratorMain - Generation ended

BUILD SUCCESSFUL Total time: 1 second

legsem commented 9 years ago

Maybe it is an effect of bad formatting in this Github thread but are you starting your COBOL statements at column 1 or at column 8?

alonsnir commented 9 years ago

yes, i believe it starts at 8. could you please take a look at the screenshot? screen shot 2015-03-29 at 5 03 32 pm

alonsnir commented 9 years ago

sorry, i was not reading the error careful and this time after correction of the column it's different:

Alons-MacBook-Pro:legstar.avro.distrib-0.4.3-SNAPSHOT alonsnir$ ant Buildfile: /Users/alonsnir/work/legstar.avro.distrib-0.4.3-SNAPSHOT/build.xml

init:

generate: [java] 0 [main] INFO com.legstar.base.generator.AbstractCob2JavaGeneratorMain - Generation started [java] 19 [main] INFO com.legstar.avro.generator.Cob2AvroGeneratorMain - Processing COBOL file /Users/alonsnir/work/legstar.avro.distrib-0.4.3-SNAPSHOT/samples/cobol/book-redefine.cob [java] 363 [main] INFO com.legstar.avro.generator.Cob2AvroGenerator - COBOL to Avro generation for: /Users/alonsnir/work/legstar.avro.distrib-0.4.3-SNAPSHOT/samples/cobol/book-redefine.cob [java] 722 [main] ERROR com.legstar.avro.generator.Cob2AvroGenerator - COBOL to Avro generation failed for: book-redefine [java] org.apache.avro.AvroRuntimeException: Duplicate in union:string [java] at org.apache.avro.Schema$UnionSchema.(Schema.java:789) [java] at org.apache.avro.Schema.parse(Schema.java:1273) [java] at org.apache.avro.Schema.parse(Schema.java:1203) [java] at org.apache.avro.Schema$Parser.parse(Schema.java:965) [java] at org.apache.avro.Schema$Parser.parse(Schema.java:953) [java] at com.legstar.avro.generator.Cob2AvroGenerator.avroCompile(Cob2AvroGenerator.java:265) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generate(Cob2AvroGenerator.java:149) [java] at com.legstar.avro.generator.Cob2AvroGenerator.generate(Cob2AvroGenerator.java:95) [java] at com.legstar.avro.generator.Cob2AvroGeneratorMain.generate(Cob2AvroGeneratorMain.java:42) [java] at com.legstar.base.generator.AbstractCob2JavaGeneratorMain.execute(AbstractCob2JavaGeneratorMain.java:106) [java] at com.legstar.avro.generator.Cob2AvroGeneratorMain.main(Cob2AvroGeneratorMain.java:28) [java] 724 [main] INFO com.legstar.base.generator.AbstractCob2JavaGeneratorMain - Generation ended

BUILD SUCCESSFUL Total time: 1 second

legsem commented 9 years ago

Ah, now that is expected.

This is no longer a problem parsing your COBOL. What is happening here is that legstar translates REDEFINES into avro unions.

Avro unions do not support the case where 2 of the alternatives are of the same type (in this case, there are 2 strings).

alonsnir commented 9 years ago

I see, please close this issue.

Thank you for looking into it.