moov-io / achgateway

Payment gateway enabling automated ACH operations in a distributed and fault tolerant way.
https://moov-io.github.io/achgateway/
Apache License 2.0
52 stars 19 forks source link

Pipeline chunk files benchmark #227

Closed adamdecaf closed 4 months ago

adamdecaf commented 4 months ago

Upgrading to moov-io/ach v1.35.0 as well, which has some performance improvements.

Without the moov-io/ach upgrade this benchmark reports

Benchmark_chunkFilesTogether
    merging_test.go:172: merged 10000 files
Benchmark_chunkFilesTogether/run
    merging_test.go:191: 3 files after merge
Benchmark_chunkFilesTogether/run-16                    1    7218584114 ns/op    230712568 B/op    871031 allocs/op
PASS
ok      github.com/moov-io/achgateway/internal/pipeline 14.087s

After upgrading moov-io/ach

Benchmark_chunkFilesTogether
    merging_test.go:172: merged 10000 files
Benchmark_chunkFilesTogether/run
    merging_test.go:191: 3 files after merge
Benchmark_chunkFilesTogether/run-16                    1    7234198724 ns/op    229498080 B/op    700987 allocs/op
PASS
ok      github.com/moov-io/achgateway/internal/pipeline 14.208s

There's still a lot of room for improvement in the merge files routine, but adding Filename to FileUploaded shouldn't hinder uploads.

codecov-commenter commented 4 months ago

Codecov Report

Attention: Patch coverage is 25.00000% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 47.63%. Comparing base (dc6e821) to head (8365ce6).

Files Patch % Lines
internal/pipeline/merging.go 25.00% 11 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #227 +/- ## ========================================== - Coverage 47.70% 47.63% -0.08% ========================================== Files 97 97 Lines 4366 4377 +11 ========================================== + Hits 2083 2085 +2 - Misses 1952 1960 +8 - Partials 331 332 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.