search

moov-io / ach

ACH implements a reader, writer, and validator for Automated Clearing House (ACH) files. The HTTP server is available in a Docker image and the Go package is available.
https://moov-io.github.io/ach/
Apache License 2.0
446 stars 149 forks source link

feat: MergeDir #1395

Closed adamdecaf closed 3 months ago

adamdecaf commented 3 months ago

MergeDir is a more performant version of consolidating ACH files together. It concurrently reads files rather than reading all files into memory.

Benchmark Results ``` $ gotest . -run BenchmarkMerge -bench BenchmarkMergeFiles/Merge goos: darwin goarch: amd64 pkg: github.com/moov-io/ach cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz BenchmarkMergeFiles/MergeFiles-16 1375 888429 ns/op 56891 B/op 637 allocs/op --- BENCH: BenchmarkMergeFiles/MergeFiles-16 merge_bench_test.go:193: 1 files merged into 1 files merge_bench_test.go:193: 100 files merged into 1 files merge_bench_test.go:193: 1375 files merged into 9 files BenchmarkMergeFiles/MergeFiles_ValidateOpts-16 1413 967463 ns/op 56931 B/op 637 allocs/op --- BENCH: BenchmarkMergeFiles/MergeFiles_ValidateOpts-16 merge_bench_test.go:197: 1 files merged into 1 files merge_bench_test.go:197: 100 files merged into 1 files merge_bench_test.go:197: 1413 files merged into 9 files BenchmarkMergeFiles/MergeDir-16 3553 300769 ns/op 57383 B/op 641 allocs/op --- BENCH: BenchmarkMergeFiles/MergeDir-16 merge_bench_test.go:215: merged 1 files into 1 files merge_bench_test.go:215: merged 100 files into 1 files merge_bench_test.go:215: merged 3553 files into 22 files BenchmarkMergeFiles/MergeDir_ValidateOpts-16 3328 308202 ns/op 57437 B/op 641 allocs/op --- BENCH: BenchmarkMergeFiles/MergeDir_ValidateOpts-16 merge_bench_test.go:233: merged 1 files into 1 files merge_bench_test.go:233: merged 100 files into 1 files merge_bench_test.go:233: merged 3328 files into 20 files BenchmarkMergeFiles/MergeFiles_3Groups-16 1095 1096058 ns/op 62565 B/op 697 allocs/op --- BENCH: BenchmarkMergeFiles/MergeFiles_3Groups-16 merge_bench_test.go:237: 1 files merged into 1 files merge_bench_test.go:237: 100 files merged into 1 files merge_bench_test.go:237: 1095 files merged into 7 files BenchmarkMergeFiles/MergeFiles_5Groups-16 1137 1105231 ns/op 62569 B/op 697 allocs/op --- BENCH: BenchmarkMergeFiles/MergeFiles_5Groups-16 merge_bench_test.go:240: 1 files merged into 1 files merge_bench_test.go:240: 100 files merged into 1 files merge_bench_test.go:240: 1137 files merged into 7 files BenchmarkMergeFiles/MergeFiles_10Groups-16 1153 1171076 ns/op 62708 B/op 697 allocs/op --- BENCH: BenchmarkMergeFiles/MergeFiles_10Groups-16 merge_bench_test.go:243: 1 files merged into 1 files merge_bench_test.go:243: 100 files merged into 1 files merge_bench_test.go:243: 1153 files merged into 7 files BenchmarkMergeFiles/MergeFiles_100Groups-16 1348 1094977 ns/op 62216 B/op 699 allocs/op --- BENCH: BenchmarkMergeFiles/MergeFiles_100Groups-16 merge_bench_test.go:246: 1 files merged into 1 files merge_bench_test.go:246: 100 files merged into 1 files merge_bench_test.go:246: 1348 files merged into 9 files PASS ok github.com/moov-io/ach 23.374s ```
codecov-commenter commented 3 months ago

Codecov Report

Merging #1395 (5f98d2b) into master (14207d2) will decrease coverage by 0.66%. The diff coverage is 20.68%.

:exclamation: Current head 5f98d2b differs from pull request most recent head 7dc3bbe. Consider uploading reports for the commit 7dc3bbe to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #1395 +/- ## ========================================== - Coverage 84.69% 84.03% -0.66% ========================================== Files 82 82 Lines 8205 8275 +70 ========================================== + Hits 6949 6954 +5 - Misses 891 955 +64 - Partials 365 366 +1 ```