search

jf-tech / omniparser

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
MIT License
931 stars 68 forks source link

Edi parsing failing, with error segment needs min occur 1, but only got 0 #142

Closed rohanbr closed 3 years ago

rohanbr commented 3 years ago

I am not able to parse the EDI. The code for the issue can be replicated from this repo https://github.com/rohanbr/edi-transformer/tree/feat/edi_parser. when i start go application, i am getting error segment no.1 (char[1,1]): segment 'ISA' needs min occur 1, but only got 0

jf-tech commented 3 years ago

@rohanbr how urgent is it? I am in the process of setting up my new dev box so might take a day or two before I will be able to take a look the issue.

One guess is if the edi text file content appears correct to you (which I haven't looked at yet) then use some binary tool to check the encoding of the text file. Is it utf-8, or some Windows based encoding? Does it contain BOM marker at the beginning of the file?

rohanbr commented 3 years ago

Its not urgent, but will be helpful if solved in 1-2 days. I checked the file, it doesn't has BOM marker at the beginning of the file

jf-tech commented 3 years ago

@rohanbr checked your sample.log input file, this EDI file as well as your schema.json have several issues:

  1. In the schema.json, you have "segment_delimiter": "|", while clearly in your sample.log the segment delimiter is \n.
  2. Assuming issue 1 is fixed, then you need to remove this line from schema.json: "ignore_crlf": true,
  3. Even after fixing both 1 and 2 in your schema, your sample.log edi file appears to be truncated (incomplete), thus the parsing and transform will not succeed.

For more EDI schema authoring, check the detailed doc here.

jf-tech commented 3 years ago

@rohanbr any more questions, or the issue can be closed?

jf-tech commented 3 years ago

Closed as answered.