search

jf-tech / omniparser

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
MIT License
971 stars 70 forks source link

Parser only loops 10 times #141

Closed tommyz4 closed 3 years ago

tommyz4 commented 3 years ago

I can not seem to get this to loop EB more than 10 times. Any help would be appreciated . 

{
    "parser_settings": {
        "version": "omni.2.1",
        "file_format_type": "edi"
    },
    "file_declaration": {
        "segment_delimiter": "~",
        "element_delimiter": "*",
        "component_delimiter": "|",
        "ignore_crlf": true,
        "segment_declarations": [
            {
                "name": "ISA",
                "child_segments": [
                    {
                        "name": "GS",
                        "child_segments": [
                            {
                                "name": "transaction_set_id",
                                "type": "segment_group",
                                "is_target": true,
                                "child_segments": [
                                    {
                                        "name": "ST",
                                        "elements": [
                                            {
                                                "name": "X12Form",
                                                "index": 1
                                            },
                                            {
                                                "name": "TransactionSetControlNumber ",
                                                "index": 2
                                            },
                                            {
                                                "name": "ImplementationConventionReference",
                                                "index": 3
                                            }
                                        ]
                                    },
                                    {
                                        "name": "BHT",
                                        "min": 0,
                                        "elements": [
                                            {
                                                "name": "BHT01",
                                                "index": 1
                                            },
                                            {
                                                "name": "BHT04",
                                                "index": 3
                                            },
                                            {
                                                "name": "BHT05",
                                                "index": 4
                                            },
                                            {
                                                "name": "BHT06",
                                                "index": 5
                                            }
                                        ]
                                    },
                                    {
                                        "name": "HL",
                                        "type": "segment_group",
                                        "min": 0,
                                        "max": -1,
                                        "child_segments": [
                                            {
                                                "name": "HL",
                                                "elements": [
                                                    {
                                                        "name": "HL1",
                                                        "index": 1
                                                    },
                                                    {
                                                        "name": "HL4",
                                                        "index": 4
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "TRN",
                                                "min": 0,
                                                "elements": [
                                                    {
                                                        "name": "TRN00",
                                                        "index": 1
                                                    },
                                                    {
                                                        "name": "TRN01",
                                                        "index": 2
                                                    },
                                                    {
                                                        "name": "TRN02",
                                                        "index": 3
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "NM1",
                                                "min": 0,
                                                "elements": [
                                                    {
                                                        "name": "NM1101",
                                                        "index": 1
                                                    },
                                                    {
                                                        "name": "NM1102",
                                                        "index": 2
                                                    },
                                                    {
                                                        "name": "NM1103",
                                                        "index": 3
                                                    },
                                                    {
                                                        "name": "NM1108 ",
                                                        "index": 8
                                                    },
                                                    {
                                                        "name": "NM1109",
                                                        "index": 9
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "N3",
                                                "min": 0,
                                                "elements": [
                                                    {
                                                        "name": "N301",
                                                        "index": 1
                                                    },
                                                    {
                                                        "name": "N302",
                                                        "index": 2
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "N4",
                                                "min": 0,
                                                "elements": [
                                                    {
                                                        "name": "N401",
                                                        "index": 1
                                                    },
                                                    {
                                                        "name": "N402",
                                                        "index": 2
                                                    },
                                                    {
                                                        "name": "N403",
                                                        "index": 3
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "DMG",
                                                "min": 0,
                                                "elements": [
                                                    {
                                                        "name": "DMG01",
                                                        "index": 1
                                                    },
                                                    {
                                                        "name": "DMG02",
                                                        "index": 2
                                                    },
                                                    {
                                                        "name": "DMG03",
                                                        "index": 3
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "DTP",
                                                "min": 0,
                                                "elements": [
                                                    {
                                                        "name": "DTP01",
                                                        "index": 1
                                                    }
                                                ]
                                            }
                                        ]
                                    },
                                    {
                                        "name": "EB",
                                        "min": 0,
                                        "max": -1,
                                        "type": "segment_group",
                                        "child_segments": [
                                            {
                                                "name": "EB",
                                                "elements": [
                                                    {
                                                        "name": "EB01",
                                                        "index": 1,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "EB02",
                                                        "index": 2,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "EB03",
                                                        "index": 3,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "EB04",
                                                        "index": 4,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "EB05",
                                                        "index": 5,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "EB06",
                                                        "index": 6,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "EB07",
                                                        "index": 7,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "EB08",
                                                        "index": 8,
                                                        "default": ""
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "DTP",
                                                "min": 0, "max": -1,
                                                "elements": [
                                                    {
                                                        "name": "DTP01",
                                                        "index": 1,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "DTP02",
                                                        "index": 2,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "DTP03",
                                                        "index": 3,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "DTP04",
                                                        "index": 4,
                                                        "default": ""
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "LS",
                                                "min": 0, "max": -1,
                                                "elements": [
                                                    {
                                                        "name": "LS01",
                                                        "index": 1,
                                                        "default": ""
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "HSD",
                                                "min": 0,
                                                "max": -1,
                                                "type": "segment_group",
                                                "child_segments": [
                                                    {
                                                        "name": "HSD",
                                                        "min": 0,
                                                        "elements": [
                                                            {
                                                                "name": "HSD01",
                                                                "index": 1,
                                                                "default": ""
                                                            },
                                                            {
                                                                "name": "HSD02",
                                                                "index": 2,
                                                                "default": ""
                                                            },
                                                            {
                                                                "name": "HSD03",
                                                                "index": 3,
                                                                "default": ""
                                                            },
                                                            {
                                                                "name": "HSD04",
                                                                "index": 4,
                                                                "default": ""
                                                            },
                                                            {
                                                                "name": "HSD05",
                                                                "index": 5,
                                                                "default": ""
                                                            },
                                                            {
                                                                "name": "HSD06",
                                                                "index": 6,
                                                                "default": ""
                                                            },
                                                            {
                                                                "name": "HSD07",
                                                                "index": 7,
                                                                "default": ""
                                                            },
                                                            {
                                                                "name": "HSD08",
                                                                "index": 8,
                                                                "default": ""
                                                            }
                                                        ]
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "MSG",
                                                "min": 0,
                                                "elements": [
                                                    {
                                                        "name": "MSG01",
                                                        "index": 1,
                                                        "default": ""
                                                    },
                                                    {
                                                        "name": "MSG02",
                                                        "index": 2,
                                                        "default": ""
                                                    }
                                                ]
                                            }
                                        ]
                                    }
                                ]
                            }
                        ]
                    }
                ]
            },
            {
                "name": "SE",
                "min": 0
            },
            {
                "name": "IEA",
                "min": 0
            }
        ]
    },
    "transform_declarations": {
        "FINAL_OUTPUT": {
            "object": {
                "transaction_set_id1": {
                    "xpath": "ST/X12Form"
                },
                "transaction_set_id2": {
                    "xpath": "ST/TransactionSetControlNumber"
                },
                "transaction_set_id3": {
                    "xpath": "ST/ImplementationConventionReference"
                },
                "BHT01": {
                    "xpath": "BHT/BHT01"
                },
                "BHT04": {
                    "xpath": "BHT/BHT04"
                },
                "BHT05": {
                    "xpath": "BHT/BHT05"
                },
                "BHT06": {
                    "xpath": "BHT/BHT06"
                },
                "HL": {
                    "array": [
                        {
                            "xpath": "HL",
                            "object": {
                                "HL1": {
                                    "xpath": "HL/HL1"
                                },
                                "HL4": {
                                    "xpath": "HL/HL4"
                                },
                                "TRN": {
                                    "xpath": "TRN/TRN00"
                                },
                                "TRN1": {
                                    "xpath": "TRN/TRN01"
                                },
                                "TRN2": {
                                    "xpath": "TRN/TRN02"
                                },
                                "NM1": {
                                    "xpath": "NM1/NM1101"
                                },
                                "NM2": {
                                    "xpath": "NM1/NM1102"
                                },
                                "NM3": {
                                    "xpath": "NM1/NM1103"
                                },
                                "NM8": {
                                    "xpath": "NM1/NM1108"
                                },
                                "NM9": {
                                    "xpath": "NM1/NM1109"
                                },
                                "N3": {
                                    "xpath": "N3/N301"
                                },
                                "N302": {
                                    "xpath": "N3/N302"
                                },
                                "N4": {
                                    "xpath": "N4/N401"
                                },
                                "N402": {
                                    "xpath": "N4/N402"
                                },
                                "N403": {
                                    "xpath": "N4/N403"
                                },
                                "DMG01": {
                                    "xpath": "DMG01/DMG01"
                                },
                                "DM02": {
                                    "xpath": "DMG02/DMG02"
                                },
                                "DMG03": {
                                    "xpath": "DMG03/DMG03"
                                },
                                "DMG04": {
                                    "xpath": "DMG04/DMG04"
                                }
                            }
                        }
                    ]
                },
                "EB": {
                    "array": [
                        {
                            "xpath": "EB",
                            "object": {
                                "EB": {
                                    "xpath": "EB/EB01"
                                },
                                "EB2": {
                                    "xpath": "EB/EB02"
                                },
                                "EB3": {
                                    "xpath": "EB/EB03"
                                },
                                "EB4": {
                                    "xpath": "EB/EB04"
                                },
                                "EB5": {
                                    "xpath": "EB/EB05"
                                },
                                "DTP1": {
                                    "xpath": "DTP/DTP01"
                                },
                                "DTP2": {
                                    "xpath": "DTP/DTP02"
                                },
                                "DTP3": {
                                    "xpath": "DTP/DTP03"
                                },
                                "HSD": {
                                    "array": [
                                        {
                                            "xpath": "HSD",
                                            "object": {
                                                "HSD1": {
                                                    "xpath": "HSD/HSD01"
                                                },
                                                "HSD2": {
                                                    "xpath": "HSD/HSD02"
                                                },
                                                "HSD3": {
                                                    "xpath": "HSD/HSD03"
                                                },
                                                "HSD4": {
                                                    "xpath": "HSD/HSD04"
                                                },
                                                "HSD5": {
                                                    "xpath": "HSD/HSD05"
                                                },
                                                "HSD6": {
                                                    "xpath": "HSD/HSD06"
                                                },
                                                "HSD7": {
                                                    "xpath": "HSD/HSD07"
                                                },
                                                "HSD8": {
                                                    "xpath": "HSD/HSD08"
                                                }
                                            }
                                        }
                                    ]
                                },
                                "MSG1": {
                                    "xpath": "MSG/MSG01"
                                },
                                "MSG2": {
                                    "xpath": "MSG/MSG02"
                                },
                                "LS": {
                                    "xpath": "LS/LS01"
                                }
                            }
                        }
                    ]
                }
            }
        }
    }
}
ISA*00* *00* *ZZ*CMS *ZZ*SUBMITTERID *171104*0734*^*00501*111111111*0*P*|~
GS*HB*CMS*SUBMITTERID*20171104*07340000*1*X*005010X279A1~
ST*271*0001*005010X279A1~
BHT*0022*11*TRANSA*20171104*07342355~
HL*1**20*1~
NM1*PR*2*CMS*****PI*CMS~
HL*2*1*21*1~
NM1*1P*2*IRNAME*****XX*1234567893~
HL*3*2*22*0~
TRN*2*TRACKNUM*ABCDEFGHIJ~
NM1*IL*1*LNAME*FNAME*M***MI*123456789A~
N3*ADDRESSLINE1*ADDRESSLINE2~
N4*CITY*ST*ZIPCODE~
DMG*D8*19400401*F~
DTP*307*RD8*20170101-20171204~
EB*6**30~
DTP*307*RD8*20170101-20170108~
EB*I**41^54~
EB*1**88~
EB*1**30^10^42^45^48^49^69^76^83^A5^A7^AG^BT^BU^BV*MA~
DTP*291*D8*20050401~
EB*D**30*MA~
DTP*292*RD8*20170116-20170120~
EB*C**30*MA**26*1316~
DTP*291*RD8*20170101-20171231~
EB*C**30*MA**29*1316~
DTP*291*RD8*20170101-20171231~
EB*C**30*MA**29*0~
DTP*291*RD8*20170116-20170120~
EB*C**42^45*MA**26*0~
DTP*292*RD8*20170101-20171231~
EB*B**30*MA**26*0~
HSD***DA**30*0~
HSD***DA**31*60~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**30*MA**7*329~
HSD***DA**30*60~
HSD***DA**31*90~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**30*MA**26*0~
HSD***DA**29*60~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**30*MA**7*329~
HSD***DA**29*30~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**30*MA**26*0~
HSD***DA**29*56~
HSD*****26*1~
DTP*435*RD8*20170116-20170120~
EB*B**30*MA**7*329~
HSD***DA**29*30~
HSD*****26*1~
DTP*435*RD8*20170116-20170120~
EB*B**AG*MA**26*0~
HSD***DA**30*0~
HSD***DA**31*20~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**AG*MA**7*164.50~
HSD***DA**30*20~
HSD***DA**31*100~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**AG*MA**26*0~
HSD***DA**29*20~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**AG*MA**7*164.50~
HSD***DA**29*80~
HSD*****26*1~
DTP*435*RD8*20170101-20171231~
EB*B**AG*MA**26*0~
HSD***DA**29*16~
HSD*****26*1~
DTP*435*RD8*20170116-20170120~
EB*B**AG*MA**7*164.50~
HSD***DA**29*80~
HSD*****26*1~
DTP*435*RD8*20170116-20170120~
EB*K**30*MA**32***DY*60~
EB*K**30*MA**33***DY*58~
EB*K**30*MA**7*658~
DTP*435*RD8*20170101-20171231~
EB*K**A7*MA**32***DY*190~
EB*K**A7*MA**33***DY*180~
EB*D**45*MA**26***99*1~
EB*1**30^2^3^5^10^14^23^24^25^26^27^28^33^36^37^38^39^40^42^50^51^52^53^67^69^73^76^83^86^98^A4^A6^
A8^AD^AE^AF^AI^AJ^AK^AL^BF^BG^BT^BU^BV^DM^UC*MB~
DTP*291*D8*20050401~
EB*C**30*MB**23*183~
DTP*291*RD8*20170101-20171231~
EB*C**30*MB**29*0~
DTP*291*RD8*20170101-20171231~
EB*A**30*MB**27**.2~
DTP*291*RD8*20170101-20171231~
EB*C**42^67^AJ*MB**23*0~
DTP*292*RD8*20170101-20171231~
EB*A**42^67^AJ*MB**27**0~
DTP*292*RD8*20170101-20171231~
EB*C***MB**23*0******HC|80061~
DTP*292*D8*20171104~
EB*A***MB**27**0*****HC|80061~
DTP*292*D8*20171104~
EB*D***MB*********HC|80061~
DTP*348*D8*20130105~
EB*D***MB*********HC|G0117~
DTP*348*D8*20120107~
EB*F**67*MB**22***VS*8~
HSD*VS*6***29~
EB*D**AD*MB***200~
DTP*292*RD8*20170101-20171231~
MSG*USED AMOUNT~
EB*D**AE*MB***0~
DTP*292*RD8*20170101-20171231~
MSG*USED AMOUNT~
EB*F**BF*MB**29***CA*72~
MSG*Professional~
EB*F**BF*MB**29***CA*72~
MSG*Technical~
EB*F**BG*MB*****99*0~
MSG*Professional~
EB*F**BG*MB*****99*0~
MSG*Technical~
EB*F**BG*MB*****99*15~
MSG*Intensive Cardiac Rehabilitation – Professional~
EB*F**BG*MB*****99*15~
MSG*Intensive Cardiac Rehabilitation – Technical~
EB*X**42***26~
DTP*472*RD8*20161222-20170116~
LS*2120~
NM1*PR*2*ORGNAME*****PI*CONTR~
NM1*1P*2******XX*1234567890~
LE*2120~
EB*X************HC|G0180~
DTP*193*D8*20140101~
EB*X************HC|G0179~
DTP*193*D8*20140501~
DTP*193*D8*20140301~
EB*X**45*MA**26~
DTP*292*RD8*20170201-20170301~
MSG*Revocation Code – 1~
LS*2120~
NM1*1P*2******XX*1234567890~
LE*2120~
EB*D**14*MB~
DTP*356*D8*20110601~
DTP*096*D8*20130105~
EB*E**10***23***DB*3~
HSD*FL*2***29~
DTP*292*RD8*20170101-20171231~
EB*R**88*OT~
REF*18*S0000 999~
DTP*292*D8*20130101~
LS*2120~
NM1*PRP*2*ORGNAME~
N3*ADDRESSLINE1*ADDRESSLINE2~
N4*CITY*ST*ZIPCODE~
PER*IC**TE*AAABBBCCCC*UR*www.website.com~
LE*2120~
EB*R**30*IN~
REF*18*H0000 999~
DTP*290*D8*20090101~
MSG*MCO Bill Option Code- C~
LS*2120~
NM1*PRP*2*ORGNAME~
N3*ADDRESSLINE1*ADDRESSLINE2~
N4*CITY*ST*ZIPCODE~
PER*IC**TE*AAABBBCCCC*UR*www.website.com~
LE*2120~
EB*R**30*13~
REF*IG*GROUPCOVERAGEPLANPOLICYNUMBER~
DTP*290*RD8*20110601-20170601~
LS*2120~
NM1*PRP*2*ORGNAME~
N3*ADDRESSLINE1*ADDRESSLINE2~
N4*CITY*ST*ZIPCODE~
LE*2120~
SE*181*0001~
GE*1*1~
IEA*1*111111111~
jf-tech commented 3 years ago

@tommyz4 several questions:

line 32 to 36: DTP is after HSD in txt but in schema HSD is after DTP
line 132 to 137: where the NM1 and LE come from, don't see them inside seg_group EB.
line 156: REF isn't defined in schema?
line 162: PER isn't defined in schema?
tommyz4 commented 3 years ago

Thank you for the feedback. It led me right to the issue. @jf-tech

jf-tech commented 3 years ago

Will keep this issue open - as I noticed the provided schema and input cause a deadlock in the edi parser, not good. In any situation omniparser should report error instead of being stuck.