Project Documentation · API Endpoints · API Guide · Community · Blog

moov-io/metro2

Moov's mission is to give developers an easy way to create and integrate bank processing into their own software products. Our open source projects are each focused on solving a single responsibility in financial services and designed around performance, scalability, and ease-of-use.

Metro2 implements a reader, writer, and validator for consumer credit history reports in an HTTP server and Go library. The HTTP server is available in a Docker image and the Go package github.com/moov-io/metro2 is available.

Project status
Usage
- As an API
- Docker
- Google Cloud
- Data persistence
- As a command line tool
- As a Package
Learn about Metro 2
Getting help
Supported and tested platforms
Contributing
Related projects

Project status

Moov Metro2 is actively used in multiple production environments. Please star the project if you are interested in its progress. If you have layers above Metro2 to simplify tasks, perform business operations, or found bugs we would appreciate an issue or pull request. Thanks!

Usage

The Metro2 project implements an HTTP server and Go library for creating and modifying files in Metro 2 format, which is used for consumer credit history reporting by U.S. credit bureaus.

Docker

We publish a public Docker image moov/metro2 on Docker Hub with each tagged release of Metro2. No configuration is required to serve on :8080. We also have Docker images for OpenShift published as quay.io/moov/metro2.

Pull & start the Docker image:

docker pull moov/metro2:latest
docker run -p 8080:8080 moov/metro2:latest

Upload a file and validate it:

curl -X POST --form "file=@./test/testdata/packed_file.json" http://localhost:8080/validator

valid file

Convert a file from JSON to raw Metro format:

curl -X POST --form "file=@./test/testdata/packed_file.json" --form "format=metro" http://localhost:8080/convert

0480HEADER                           555555555508202002051019990510199905101999YOUR BUSINESS NAME HERE                 LINE ONE OF YOUR ADDRESS LINE TWO OF YOUR ADDRESS LINE THERE OF YOUR ADDRESS                    1234567890
...

Convert a file from raw Metro format to JSON:

curl -X POST --form "file=@./test/testdata/packed_file.dat" --form "format=json" http://localhost:8080/convert

{"header": {"blockDescriptorWord": 370,"recordDescriptorWord": 366,"recordIdentifier": "HEADER","transUnionProgramIdentifier": "5555555555","activityDate": "2002-08-20T00:00:00Z", ...

Google Cloud Run

To get started in a hosted environment you can deploy this project to the Google Cloud Platform.

From your Google Cloud dashboard create a new project and call it:

moov-metro2-demo

Enable the Container Registry API for your project and associate a billing account if needed. Then, open the Cloud Shell terminal and run the following Docker commands, substituting your unique project ID:

docker pull moov/metro2
docker tag moov/metro2 gcr.io/<PROJECT-ID>/metro2
docker push gcr.io/<PROJECT-ID>/metro2

Deploy the container to Cloud Run:

gcloud run deploy --image gcr.io/<PROJECT-ID>/metro2 --port 8080

Select your target platform to 1, service name to metro2, and region to the one closest to you (enable Google API service if a prompt appears). Upon a successful build you will be given a URL where the API has been deployed:

https://YOUR-METRO2-APP-URL.a.run.app

Now you can complete a health check:

curl https://YOUR-METRO2-APP-URL.a.run.app/health

You should get this response:

{"status":"alive"}

Data persistence

By design, Metro2 does not persist (save) any data about the files or entry details created. The only storage occurs in memory of the process and upon restart Metro2 will have no files or data saved. Also, no in-memory encryption of the data is performed.

Command line

Metro2 has a command line interface to manage Metro 2 files and launch a web service.

metro2 --help

Usage:
   [command]

Available Commands:
  convert     Convert metro file format
  help        Help about any command
  print       Print metro file
  validator   Validate metro file
  web         Launches web server

Flags:
  -h, --help           help for this command
      --input string   input file (default is $PWD/metro.json)

Use " [command] --help" for more information about a command.

Each interaction that the library supports is exposed in a command-line option:

Command	Info
`convert`	The convert command allows users to convert a metro file to a specified file format (json, metro). The result will create a new file.
`print`	The print command allows users to print a metro file in a specified file format (json, metro).
`validator`	The validator command allows users to validate a metro file.
`web`	The web command will launch a web server with endpoints to manage metro files.

Go library

This project uses Go Modules and Go v1.18 or newer. See Golang's install instructions for help setting up Go. You can download the source code and we offer tagged and released versions as well. We highly recommend you use a tagged release for production.

Package Installation

$ git@github.com:moov-io/metro2.git

# Pull down into the Go Module cache
$ go get -u github.com/moov-io/metro2

$ go doc github.com/moov-io/metro2

Package Overview

Package file defines main interfaces to a metro file.

func NewFile(format string) (File, error)

func NewFileFromReader(r io.Reader) (File, error)

type File
    GetType() string
    SetType(string) error
    SetRecord(lib.Record) error
    AddDataRecord(lib.Record) error
    GetRecord(string) (lib.Record, error)
    GetDataRecords() []lib.Record
    GeneratorTrailer() (lib.Record, error)
    Parse(record string) error
    String(newline bool) string
    Validate() error

File has some records for header, body, trailer

Records have distinguished by file type

type Record
    Name() string
    Parse(string) (int, error)
    String() string
    Validate() error
    Length() int
    BlockSize() int
    AddApplicableSegment(Segment) error
    GetSegments(string) []Segment

type BaseSegment        // for character file format

type PackedBaseSegment  // for packed file format

type HeaderRecord       // for character file format

type PackedHeaderRecord // for packed file format

type TrailerRecord      // for character file format

type PackedTrailerRecord // for packed file format

Record has some segments to describe special form

Base Record can use correctly the features of appendable segments

type Record
    Name() string
    Parse(string) (int, error)
    String() string
    Validate() error
    Length() int
    BlockSize() int
    AddApplicableSegment(Segment) error
    GetSegments(string) []Segment

type J1Segment

type J2Segment

type K1Segment

type K2Segment

type K3Segment

type K4Segment

type L1Segment

type N1Segment

Package Examples

File object has type for metro packed file and metro character file, should specify file type when creating new file

f, err := file.NewFile(utils.PackedFileFormat)
err = json.Unmarshal([]byte(buf), f)

File object will create from metro file buffer directly, CreateFile function will create new file and will set file type based on metro file body

r, _ = os.Open(filepath.Join("...))
f, err := file.NewFileFromReader(r)

File object will manage records using member functions

Please use exact record object that has same type (header, base, trailer)

f, err := file.NewFile(utils.CharacterFileFormat)
newHeader := lib.HeaderRecord{}
err = f.SetRecord(&newSegment)
exsitedHeader, err := f.GetRecord(utils.HeaderRecordName)
newBase := lib.HeaderRecord{}
...

Similarly, Record can manage the appendable segments

Learn about Metro 2

Getting help

channel	info
Twitter @moov	You can follow Moov.io's Twitter feed to get updates on our project(s). You can also tweet us questions or just share blogs or stories.
GitHub Issue	If you are able to reproduce a problem please open a GitHub Issue under the specific project that caused the error.
moov-io slack	Join our slack channel to have an interactive discussion about the development of the project.

Supported and tested platforms

64-bit Linux (Ubuntu, Debian), macOS, and Windows

Contributing

Yes please! Please review our Contributing guide and Code of Conduct to get started!

Releasing

To make a release of metro2 simply open a pull request with CHANGELOG.md and version.go updated with the next version number and details. You'll also need to push the tag (i.e. git push origin v1.0.0) to origin in order for CI to make the release.

Testing

We maintain a comprehensive suite of unit tests and recommend table-driven testing when a particular function warrants several very similar test cases. To run all test files in the current directory, use go test. Current overall coverage can be found on Codecov.

Fuzzing

We currently run fuzzing over ACH in the form of a Github Action. Please report crashes examples to oss@moov.io. Thanks!

Related projects

As part of Moov's initiative to offer open source fintech infrastructure, we have a large collection of active projects you may find useful:

Moov Watchman offers search functions over numerous trade sanction lists from the United States and European Union.
Moov Fed implements utility services for searching the United States Federal Reserve System such as ABA routing numbers, financial institution name lookup, and FedACH and Fedwire routing information.
Moov Wire implements an interface to write files for the Fedwire Funds Service, a real-time gross settlement funds transfer system operated by the United States Federal Reserve Banks.
Moov ACH provides ACH file generation and parsing, supporting all Standard Entry Codes for the primary method of money movement throughout the United States.
Moov Image Cash Letter implements Image Cash Letter (ICL) files used for Check21, X.9 or check truncation files for exchange and remote deposit in the U.S.

License

Apache License 2.0 - See LICENSE for details.

moov-io / metro2

readme