Md2orchestra parsing issue

[nngakosso]

Hello I restested tablature and I dont know why I cant get m2orchestra to work properly. Even with files generated from orchestra2md, which works perfectly. I still have a blank file anyway.

I got this error on IntelliJ:

line 34:347 extraneous input '21=723|. -723 in field 12 would be mapped int as |12=-723|.                                                                                                                               |' expecting {<EOF>, NEWLINE}
14:49:50.135 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 34 position 347 due to extraneous input '21=723|. -723 in field 12 would be mapped int as |12=-723|.                                                                                                                               |' expecting {<EOF>, NEWLINE}
line 35:0 mismatched input '| Language            ' expecting TABLEDELIMINATORCELL
14:49:50.146 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 35 position 0 due to mismatched input '| Language            ' expecting TABLEDELIMINATORCELL
line 40:107 extraneous input '18=2 A F| ).                                                                                                                               |' expecting {<EOF>, NEWLINE}
14:49:50.181 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 40 position 107 due to extraneous input '18=2 A F| ).                                                                                                                               |' expecting {<EOF>, NEWLINE}
line 41:0 mismatched input '| MultipleStringValue ' expecting TABLEDELIMINATORCELL
14:49:50.194 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 41 position 0 due to mismatched input '| MultipleStringValue ' expecting TABLEDELIMINATORCELL
line 41:109 extraneous input '277=AV AN A| ).                                                                                                                               |' expecting {<EOF>, NEWLINE}
14:49:50.210 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 41 position 109 due to extraneous input '277=AV AN A| ).                                                                                                                               |' expecting {<EOF>, NEWLINE}
line 272:0 no viable alternative at input '\n'
14:49:50.534 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 272 position 0 due to no viable alternative at input '\n'
14:49:50.571 [main] FATAL io.fixprotocol.md2orchestra.Md2Orchestra - Md2Orchestra: exception occurred
java.lang.NullPointerException: null
        at io.fixprotocol.md2orchestra.RepositoryBuilder.findDatatypeByName(RepositoryBuilder.java:989) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.RepositoryBuilder.addType(RepositoryBuilder.java:501) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.RepositoryBuilder.addFieldAndType(RepositoryBuilder.java:342) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.RepositoryBuilder.addField(RepositoryBuilder.java:332) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.RepositoryBuilder.accept(RepositoryBuilder.java:111) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.RepositoryBuilder.accept(RepositoryBuilder.java:61) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md.antlr.MarkdownEventSource.exitTablerow(MarkdownEventSource.java:272) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md.antlr.MarkdownParser$TablerowContext.exitRule(MarkdownParser.java:741) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.exitRule(ParseTreeWalker.java:47) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:30) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md.event.DocumentParser.parse(DocumentParser.java:48) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:246) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:222) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:143) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.main(Md2Orchestra.java:104) [md2orchestra-0.1.0-jar-with-dependencies.jar:?]

[donmendelson]

@nngakosso it is failing to parse the markdown. Please attach a file that demonstrates the problem for troubleshooting.

[nngakosso]

The murex.md file you already have and this one I just made among several others from orchestra2md. output.zip

[donmendelson]

I see the problem with the phrase

would be mapped int as |21=723|.

The issue is that the pipe character is column delimiter in markdown tables. This causes the parser to see more columns in the line than there are column headers in that table.

Please try escaping those pipe characters with backslash as \|. See https://github.github.com/gfm/#backslash-escapes

[nngakosso]

I did it but now I got this error :

line 22723:0 no viable alternative at input '\n'
16:09:05.576 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 22723 position 0 due to no viable alternative at input '\n'
16:09:05.861 [main] FATAL io.fixprotocol.md2orchestra.Md2Orchestra - Md2Orchestra: exception occurred
java.io.IOException: Markdown parser failed; 1
        at io.fixprotocol.md.event.DocumentParser.parse(DocumentParser.java:52) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:246) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:222) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:143) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?]
        at io.fixprotocol.md2orchestra.Md2Orchestra.main(Md2Orchestra.java:104) [md2orchestra-0.1.0-jar-with-dependencies.jar:?]

[donmendelson]

Is that number the last line of the document? If so, please try deleting the last blank line of the markdown file and rerun.

If it's successful then, we'll turn this into a bug report for failure to parse a blank line at end of the document. I've encountered that issue before but I haven't resolved it.

[nngakosso]

No more error now but the file is quite short as you can see. output3.zip

[donmendelson]

I ran it after escaping the pipe characters and deleting blank lines at the end with this command line:

java -jar target\md2orchestra-0.1.0-jar-with-dependencies.jar -i output.md -r FixRepository50SP2EP247.xml -o output.xml

It produced a large file - input and output attached issue15.zip

[nngakosso]

Glad that it is working on your side, on mine I got another error I can't figure out why. I ran the same command line as usual even tried with both FixRepository50SP2EP247 and OrchestraEP256 but I got this with your file with pipe characters escaped:

11:14:40.981 [main] FATAL io.fixprotocol.md2orchestra.Md2Orchestra - Md2Orchestra: exception occurred java.lang.NumberFormatException: Zero length BigInteger at java.math.BigInteger.(BigInteger.java:485) ~[?:?] at java.math.BigInteger.(BigInteger.java:673) ~[?:?] at com.sun.xml.bind.DatatypeConverterImpl._parseInteger(DatatypeConverterImpl.java:61) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$22.parse(RuntimeBuiltinLeafInfoImpl.java:766) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$22.parse(RuntimeBuiltinLeafInfoImpl.java:764) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.reflect.TransducedAccessor$CompositeTransducedAccessorImpl.parse(TransducedAccessor.java:209) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.StructureLoader.startElement(StructureLoader.java:182) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:547) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:526) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.SAXConnector.startElement(SAXConnector.java:138) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:518) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:374) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2725) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:541) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:888) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:824) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1224) ~[?:?] at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:635) ~[?:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:228) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:199) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:170) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:209) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.RepositoryBuilder.unmarshal(RepositoryBuilder.java:946) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.RepositoryBuilder.(RepositoryBuilder.java:87) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:241) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:222) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:143) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.main(Md2Orchestra.java:104) [md2orchestra-0.1.0-jar-with-dependencies.jar:?]

C:\Users\nngakosso\Documents\tablature\md2orchestra\target>java -jar md2orchestra-0.1.0-jar-with-dependencies.jar -i output.md -r FixRepository50SP2EP247.xml -o output.xml WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. 11:26:09.808 [main] FATAL io.fixprotocol.md2orchestra.Md2Orchestra - Md2Orchestra: exception occurred java.lang.NumberFormatException: Zero length BigInteger at java.math.BigInteger.(BigInteger.java:485) ~[?:?] at java.math.BigInteger.(BigInteger.java:673) ~[?:?] at com.sun.xml.bind.DatatypeConverterImpl._parseInteger(DatatypeConverterImpl.java:61) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$22.parse(RuntimeBuiltinLeafInfoImpl.java:766) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$22.parse(RuntimeBuiltinLeafInfoImpl.java:764) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.reflect.TransducedAccessor$CompositeTransducedAccessorImpl.parse(TransducedAccessor.java:209) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.StructureLoader.startElement(StructureLoader.java:182) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:547) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:526) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.SAXConnector.startElement(SAXConnector.java:138) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:518) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:374) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2725) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112) ~[?:?] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:541) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:888) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:824) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) ~[?:?] at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1224) ~[?:?] at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:635) ~[?:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:228) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:199) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:170) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:209) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.RepositoryBuilder.unmarshal(RepositoryBuilder.java:946) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.RepositoryBuilder.(RepositoryBuilder.java:87) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:241) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:222) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:143) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.main(Md2Orchestra.java:104) [md2orchestra-0.1.0-jar-with-dependencies.jar:?]

[donmendelson]

@nngakosso I reproduced the problem with EP 256. The problem was with a minor defect in the Orchestra reference file that prevented it from being parsed - an id attribute was blank, so it failed to parse as numeric. This was corrected in EP 257 which can be obtained from fixtrading.org or from GitHub as FIX Latest. When I used that file, md2orchestra ran to completion. (I think had manually corrected the previous file for testing but forgot that I had done that.)

[nngakosso]

I'm running on java 14 maybe that's why? I got this error with both your file and the murex.md file I got

C:\Users\nngakosso\Documents\tablature\md2orchestra\target>java -jar md2orchestra-0.1.0-jar-with-dependencies.jar -i output.md -r OrchestraFIXLatest.xml -o output.xml WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. line 1:0 mismatched input '' expecting {HEADINGLINE, LISTLINE, PARAGRAPHLINE, NEWLINE, CELLTEXT} 17:09:35.420 [main] ERROR io.fixprotocol.md.event.DocumentParser.SyntaxErrorListener - Markdown parser failed at line 1 position 0 due to mismatched input '' expecting {HEADINGLINE, LISTLINE, PARAGRAPHLINE, NEWLINE, CELLTEXT} 17:09:35.426 [main] FATAL io.fixprotocol.md2orchestra.Md2Orchestra - Md2Orchestra: exception occurred java.io.IOException: Markdown parser failed; 1 at io.fixprotocol.md.event.DocumentParser.parse(DocumentParser.java:52) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:246) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:222) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.generate(Md2Orchestra.java:143) ~[md2orchestra-0.1.0-jar-with-dependencies.jar:?] at io.fixprotocol.md2orchestra.Md2Orchestra.main(Md2Orchestra.java:104) [md2orchestra-0.1.0-jar-with-dependencies.jar:?]

I still try to figure out why I doesn't work for me but for you.

[nngakosso]

Those are the files I'm using you should already have it. murex.zip output.zip

[nngakosso]

I got this error:

10:55:01.722 [main] ERROR io.fixprotocol.md2orchestra.RepositoryBuilder - RepositoryBuilder unknown groupRef id; name=PtysSubGrp

But I've been able to generate the orchestra files with both files now.

[donmendelson]

Closing. Open a new issue for any specific anomalies.