jf-tech / omniparser

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
MIT License
971 stars 70 forks source link
codeless csv delimited edi edifact etl fixed-length fixed-width golang javascript json parser schema schemas streaming transform txt x12 xml

omniparser

CI codecov Go Report Card PkgGoDev Mentioned in Awesome Go

Omniparser is a native Golang ETL parser that ingests input data of various formats (CSV, txt, fixed length/width, XML, EDI/X12/EDIFACT, JSON, and custom formats) in streaming fashion and transforms data into desired JSON output based on a schema written in JSON.

Min Golang Version: 1.16

Licenses and Sponsorship

Omniparser is publicly available under MIT License. Individual and corporate sponsorships are welcome and gratefully appreciated, and will be listed in the SPONSORS page. Company-level sponsors get additional benefits and supports granted in the COMPANY LICENSE.

Documentation

Docs:

References:

Examples:

In the example folders above you will find pairs of input files and their schema files. Then in the .snapshots sub directory, you'll find their corresponding output files.

Online Playground (not functioning)

~~Use The Playground (may need to wait for a few seconds for instance to wake up) for trying out schemas and inputs, yours or existing samples, to see how ingestion and transform work.~~

As for now (2023/03/14), all of our previous free docker hosting solutions went away and we haven't found another one yet. For now please clone the repo and use ./cli.sh as described in the Getting Started page.

Why

Requirements

Recent Major Feature Additions/Changes

Footnotes