indice-co / EDI.Net

EDI Serializer/Deserializer. Supports EDIFact, X12 and TRADACOMS formats
MIT License
450 stars 170 forks source link

Parsing 820 throws an error at GS #140

Closed skhan-nacmgit closed 4 years ago

skhan-nacmgit commented 4 years ago

This a basic test:

POCO

    namespace EDITest
    {
        public class RemittanceAdvice_820
        {

        #region ISA and IEA
        [EdiValue("9(2)", Path = "ISA/0", Description = "ISA01 - Authorization Information Qualifier")]
        public int AuthorizationInformationQualifier { get; set; }

        [EdiValue("X(10)", Path = "ISA/1", Description = "ISA02 - Authorization Information")]
        public string AuthorizationInformation { get; set; }

        [EdiValue("9(2)", Path = "ISA/2", Description = "ISA03 - Security Information Qualifier")]
        public string Security_Information_Qualifier { get; set; }

        [EdiValue("X(10)", Path = "ISA/3", Description = "ISA04 - Security Information")]
        public string Security_Information { get; set; }

        [EdiValue("9(2)", Path = "ISA/4", Description = "ISA05 - Interchange ID Qualifier")]
        public string ID_Qualifier { get; set; }

        [EdiValue("X(15)", Path = "ISA/5", Description = "ISA06 - Interchange Sender ID")]
        public string Sender_ID { get; set; }

        [EdiValue("9(2)", Path = "ISA/6", Description = "ISA07 - Interchange ID Qualifier")]
        public string ID_Qualifier2 { get; set; }

        [EdiValue("X(15)", Path = "ISA/7", Description = "ISA08 - Interchange Receiver ID")]
        public string Receiver_ID { get; set; }

        [EdiValue("9(6)", Path = "ISA/8", Format = "yyMMdd", Description = "I09 - Interchange Date")]
        [EdiValue("9(4)", Path = "ISA/9", Format = "HHmm", Description = "I10 - Interchange Time")]
        public DateTime Date { get; set; }

        [EdiValue("X(1)", Path = "ISA/10", Description = "ISA11 - Interchange Control Standards ID")]
        public string Control_Standards_ID { get; set; }

        [EdiValue("9(5)", Path = "ISA/11", Description = "ISA12 - Interchange Control Version Num")]
        public int ControlVersion { get; set; }

        [EdiValue("9(9)", Path = "ISA/12", Description = "ISA13 - Interchange Control Number")]
        public int ControlNumber { get; set; }

        [EdiValue("9(1)", Path = "ISA/13", Description = "ISA14 - Acknowledgement Requested")]
        public bool? AcknowledgementRequested { get; set; }

        [EdiValue("X(1)", Path = "ISA/14", Description = "ISA15 - Usage Indicator")]
        public string Usage_Indicator { get; set; }

        [EdiValue("X(1)", Path = "ISA/15", Description = "ISA16 - Component Element Separator")]
        public char? Component_Element_Separator { get; set; }

        [EdiValue("9(1)", Path = "IEA/0", Description = "IEA01 - Num of Included Functional Grps")]
        public int GroupsCount { get; set; }

        [EdiValue("9(9)", Path = "IEA/1", Description = "IEA02 - Interchange Control Number")]
        public int TrailerControlNumber { get; set; }

        #endregion

        public List<FunctionalGroup> Groups { get; set; }

        [EdiGroup]
        public class FunctionalGroup
        {

            [EdiValue("X(2)", Path = "GS/0", Description = "GS01 - Functional Identifier Code")]
            public string FunctionalIdentifierCode { get; set; }

            [EdiValue("X(10)", Path = "GS/1", Description = "GS02 - Application Sender's Code")]
            public string ApplicationSenderCode { get; set; }

            [EdiValue("X(6)", Path = "GS/2", Description = "GS03 - Application Receiver's Code")]
            public string ApplicationReceiverCode { get; set; }

            [EdiValue("9(8)", Path = "GS/3", Format = "yyyyMMdd", Description = "GS04 - Date")]
            [EdiValue("9(4)", Path = "GS/4", Format = "HHmm", Description = "GS05 - Time")]
            public DateTime Date { get; set; }

            [EdiValue("9(9)", Path = "GS/5", Format = "HHmm", Description = "GS06 - Group Control Number")]
            public int GroupControlNumber { get; set; }

            [EdiValue("X(1)", Path = "GS/6", Format = "HHmm", Description = "GS07 Responsible Agency Code")]
            public string AgencyCode { get; set; }

            [EdiValue("X(6)", Path = "GS/7", Format = "HHmm", Description = "GS08 Version / Release / Industry Identifier Code")]
            public string Version { get; set; }
        }
     }
    }

Loading the doc:

            // Load the EDI Parser
            var grammar = EdiGrammar.NewX12();

            // Set terminators specific to vendors
            grammar.SetAdvice(new[] { '^', '*', '.', '?', ' ', '\n'});

            // Wire up an 850 MAP
            var interchange = default(RemittanceAdvice_820);

            // Load and EDI file and begin parsing
            using (var stream = new StreamReader(@"C:\Temp\856\TP\820.EDI"))
            {
                interchange = new EdiSerializer().Deserialize<RemittanceAdvice_820>(stream, grammar);

                Console.WriteLine("Document Date: " + interchange.Date);
                Console.WriteLine("Sender ID: " + interchange.Sender_ID);
                Console.WriteLine("Reveiver ID: " + interchange.Receiver_ID);

          }

This is a test 820

ISA*00*          *00*          *08*TPTESTUS00     *ZZ*TESTCOMP       *191029*1900*:*00501*820101047*0*P*>
GS*RA*TPTESTUS00*TESTCOMP*20191029*1900*820101047*X*005010
ST*820*101053
BPR*I*21298.97*C*CHK************20191029*VEN
TRN*3*4529952
CUR*PR*USD
N1*PR*MART INC.*UL*1238742000008
N1*PE*Test Company, LLC
REF*IA*446963
ENT*1
RMR*IV*26801*PO*4408.75*4650.41*97.9
REF*DP*00040
REF*MR*0033
REF*PO*1016071221
REF*MC*858008293
REF*19*07
REF*ST*0078742031897
DTM*097*20190904
ADX*-143.76*59*CM*26801
REF*6O*022
REF*MC*000000025
RMR*IV*26927*PO*4653.84*4753.92*100.08
REF*DP*00040
REF*MR*0033
REF*PO*3016440142
REF*MC*884600831
REF*19*07
REF*ST*0078742029733
DTM*097*20190910
RMR*IV*26935*PO*4413.11*4508.02*94.91
REF*DP*00040
REF*MR*0020
REF*PO*5316490001
REF*MC*884600839
REF*19*07
REF*ST*0078742038599
DTM*097*20190910
RMR*IV*27009*PO*2276.1*2323.53*47.43
REF*DP*00002
REF*MR*0033
REF*PO*5719877330
REF*MC*888309454
REF*19*07
REF*ST*0078742041360
DTM*097*20190911
RMR*IV*27018*PO*2276.1*2323.53*47.43
REF*DP*00002
REF*MR*0033
REF*PO*7514988202
REF*MC*888309461
REF*19*07
REF*ST*0078742028200
DTM*097*20190911
RMR*IV*27028*PO*1264.5*1290.85*26.35
REF*DP*00002
REF*MR*0033
REF*PO*8614957942
REF*MC*888309470
REF*19*07
REF*ST*0078742030098
DTM*097*20190911
RMR*IV*27029*PO*2006.57*2049.72*43.15
REF*DP*00040
REF*MR*0020
REF*PO*8616050842
REF*MC*888400008
REF*19*07
REF*ST*0078742028200
DTM*097*20190911
SE*68*101053
GE*1*820101047
IEA*1*820101047

The Error:

An unhandled exception of type 'indice.Edi.EdiException' occurred in indice.Edi.dll Additional information: Unable to convert string '> GS' to char. It is more than 1 character long.

skhan-nacmgit commented 4 years ago

Interestingly, a PO with the same ISA seems to work just fine.

ISA00 00 08TPTESTUS00 ZZTESTCOMP 1910291900:005018201010470P>

cleftheris commented 4 years ago

@skhan-nacmgit You are not setting the grammar correctly. The in the SetAdvise you are not providing the characters in the correct positions. Please use the other overload which guides you through like seen below:


            // Load the EDI Parser
            var grammar = EdiGrammar.NewX12();

            // Set terminators specific to vendors
            grammar.SetAdvice('*', '*', '>', '\n',  null, null, '.'); // <-- use this

Also since the segment terminator is the newline itself make sure you got a newline in the end of your transmission at the end of the IEA line

skhan-nacmgit commented 4 years ago

This suggestion doesnt work. I am setting the grammar like this:

grammar.SetAdvice('*', '*', ':', '\n', null, null, '.');

This is a sample taken from a document in prod

ISA*00*          *00*          *08*999999US00     *ZZ*TEST99         *191118*1956*:*00501*820101061*0*P*>
GS*RA*999999US00*TEST99*20191118*1956*820101061*X*005010
ST*820*101067
BPR*I*104029.25*C*CHK************20191118*VEN
TRN*3*4614814
CUR*PR*USD
N1*PR*TESTMART INC.*UL*0078992000008
N1*PE*Test Company, LLC
REF*IA*449963
ENT*1
RMR*IV*27213*PO*4212.34*4302.93*90.59
REF*DP*00040
REF*MR*0020
REF*PO*2708989667
REF*MC*944506856
REF*19*07
REF*ST*0078742033587
DTM*097*20190924
SE*336*101067
GE*1*820101061
IEA*1*820101061 
          var grammar = EdiGrammar.NewX12();
           grammar.SetAdvice('*', '*', ':', '\n', null, null, '.');
            // Wire up an 820 MAP
            var interchange = default(RemittanceAdvice_820);

            using (var stream = new StreamReader(@"C:\PATH_TO_820.304.EDI"))
            {
                interchange = new EdiSerializer().Deserialize<RemittanceAdvice_820>(stream, grammar);

                //Console.WriteLine("Document Date: " + interchange.Date);
                //Console.WriteLine("Sender ID: " + interchange.Sender_ID);
                //Console.WriteLine("Reveiver ID: " + interchange.Receiver_ID);
            }
            Console.Read();

Fails with

Additional information: Unable to convert string '>

GS' to char. It is more than 1 character long.

Evan if I add a New Line after IEA

skhan-nacmgit commented 4 years ago

And if I change the grammar to:

grammar.SetAdvice('', '', '>', '\n', null, null, '.');

I get the error: Unterminated string. Expected delimiter. Path 'ISA[1061][0]', line 412, position 17.

cleftheris commented 4 years ago

Hi you issue is definitely related to the advise characters but it does not seem to be a bug since I cannot reproduce. I am not sure of the correct advise if I don't see the original file. I suspect that by copy pasting in the github comment I only see \r\n so I suspect that \n is the correct character for segment termination. It may not be the case, and you may have the \r instead (Or even a more weird control character that is not even visible).

you can either

  1. send me the file via email if you have no sensitive info in your transmission
  2. or even better clone the repo and replace the 140 test case transmission file x12.820.issue140.edi then run the test https://github.com/indice-co/EDI.Net/blob/a6eb98ad31fc2bbe4c325015659a45c49f00ff66/test/indice.Edi.Tests/EdiTextReaderTests.cs#L976 If you manage to reproduce send me a pull request with your customized test and I will lookinto it again
skhan-nacmgit commented 4 years ago

I am setting the grammar as such: grammar.SetAdvice('*', '*', ':', '\n', null, null, '.');

and all when I choose to display all characters in my file using Notepad++ I see each line ending with CRLF

e.g.

REF*DP*00040 CRLF
REF*MR*0020 CRLF
REF*PO*2708989667 CRLF
REF*MC*944506856 CRLF
REF*19*07 CRLF
REF*ST*0078742033587 CRLF

I can cleanup a production file and email it to you if that helps.

cleftheris commented 4 years ago

Please make a pull request with your failing test & file. Because as I stated above the file is working as expected on my side.

One last thing. Are you running on the latest version of the library?

skhan-nacmgit commented 4 years ago

Ok. I'll make the pull request. I am using v1.9.6.0.

skhan-nacmgit commented 4 years ago

I copied down your test and for some reason it works. I do have to manually add the newline after IEA. Is there a way to get this added automatically if required in a file?

This issue can be closed.

cleftheris commented 4 years ago

I do have to manually add the newline after IEA. Is there a way to get this added automatically if required in a file?

There is a bug that I will be fixing today regarding the new line. Exprect to land in v1.9.7.