[Proposal]: Enhancing Ballerina EDI Tool to Handle Length Constraints in EDIs

Summary

This proposal aims to enhance the Ballerina programming language's EDI tool by introducing length constraint support for EDI files. The primary focus is on accommodating minimum and maximum length requirements for data elements in compliance with various EDI standards.

Goals

Improve the "length" parameter in the EDI schema to specify acceptable length ranges for each field.
Implement a validation mechanism during the parsing and generation processes to EDI ensure adherence to specified length constraints.
Enhance the existing length parameter to support both fixed lengths and the proposed minimum and maximum length constraints.

Non-Goals

It does not intend to modify the current definition of fixed lengths for fields.

Motivation

While the current Ballerina EDI tool effectively manages field constraints such as data type, required, and length(fixed), the absence of min/max length constraint support poses a limitation.Despite having a length constraint primarily reserved for defining fixed lengths, various EDI specifications, including widely used standards like EDIFACT, HIPAA, and ASC X12, impose specific minimum and/or maximum length requirements for data elements [1,2]. The tool should evolve to seamlessly accommodate these constraints to ensure compliance with these standards.

Success Metrics

Increased usage of the length parameter in EDI schemas.
Reduced instances of non-compliance with EDI standards related to length constraints.
Positive feedback from developers regarding the clarity of error messages and ease of troubleshooting.

Description

We propose improving the current length parameter by specifying minimum and maximum length constraints for each field.A validation mechanism will be integrated into the module-ballerina-edi package, ensuring adherence to these constraints during the parsing and generation of EDI files.

Technical Implementation

Schema Parser Modification: This length parameter will be associated with each field, specifying the acceptable length range for the corresponding data element. The tool will validate input EDI files against these constraints during parsing and generation.

Example EDI Schema Modification

{
  "name": "EDIFACTOrder",
  "delimiters": {"segment": "'", "field": "+"},
  "segments": {
    "UNH": {
      "tag": "MessageHeader",
      "fields": [
        {"tag": "MessageRefNum", "length": {"min": 1, "max": 14}},
        {"tag": "MessageType", "length": {"min": 3, "max": 3}},
        {"tag": "Version", "length": {"min": 1, "max": 3}},
        {"tag": "Release", "length": {"min": 1, "max": 3}},
        {"tag": "ControllingAgency", "length": {"min": 1, "max": 2}}
      ]
    },
    "BGM": {
      "tag": "BeginningOfMessage",
      "fields": [
        {"tag": "DocumentNameCode", "length": {"min": 3, "max": 3}},
        {"tag": "DocumentNumber", "length": {"min": 1, "max": 35}},
        {"tag": "MessageFunction", "length": {"min": 1, "max": 3}},
        {"tag": "ResponseType", "length": {"min": 1, "max": 3}},
        {"tag": "TestIndicator", "length": {"min": 1, "max": 1}}
      ]
    }
  }
}

The existing parameter will remain unchanged to define a fixed length. Users can utilize it as they did previously, as follows:

{"tag": "MessageRefNum", "length": 25}

Since the existing definition remains as it is with the proposed improvement, there will not be any breaking changes related to the existing length definitions in schemas.

Validation : The tool will integrate a validation mechanism, ensuring that each field's length adheres to the specified constraints.This will be handled through out the parsing process of toEdiString() and fromEdiString() of the module-ballerina-edi package. Clear error messages will be generated for quick issue identification and the error messages will pinpoint the field name, type of violation (exceed the max/below the min) and schema related to the violation, aiding developers in debugging and troubleshooting.
Testing: Testing suite will be developed to validate the tool's functionality with EDI files featuring length constraints, ensuring reliability and accuracy.

Alternatives

Considered alternatives include relying solely on fixed length constraints or handling length constraints outside the schema. However, these approaches were deemed less flexible and might compromise compliance with specific EDI standards.

Testing

Testing will involve the development of a comprehensive suite to validate the tool's functionality with EDI files featuring length constraints. This includes both unit tests and broader tests to ensure reliability and accuracy.

Risks and Assumptions

Potential risks include unforeseen complexities in modifying the schema parser module and potential resistance to adopting the proposed changes. Assumptions include a smooth integration process and positive reception from the developer community.

Dependencies

This proposal depends on modifications to the schema parser module (module-edi) and the integration of a validation module into the module-ballerina-edi package. Coordination with developers and community feedback is crucial for successful implementation.

References

[1] UNECE, "UN/EDIFACT Part 4 - Joint Message Design Principles," Annex B. [Online]. Available: https://unece.org/trade/uncefact/unedifact/part-4-Annex-B
[2] STEDI, "What is an Element?" Electronic Data Interchange (EDI) Essentials. [Online]. Available: https://www.stedi.com/docs/edi-essentials/x12/elements/what-is-an-element

ballerina-platform / ballerina-library