moov-io / achgateway

Payment gateway enabling automated ACH operations in a distributed and fault tolerant way.
https://moov-io.github.io/achgateway/
Apache License 2.0
54 stars 19 forks source link

feat: Endpoint to produce FileUploaded from previous cutoff #183

Closed adamdecaf closed 1 year ago

adamdecaf commented 1 year ago

When the receiving SFTP server is down during file upload the file may not be uploaded. Often a human operator will manually upload the file once the SFTP server is running again. This manual upload does not produce a FileUploaded event so transactions can be stuck waiting for a status transition.

We could add an endpoint to produce FileUploaded for a shard's isolated directory.

PUT /shards/{shardName}/pipeline/$isolatedDirectory/file-uploaded

Given an isolated directory (testing-20230602-093000) we will have a directory of files. The endpoint can produce FileUploaded as if it was processing the directory.

-rw------- 1 root root 1304 Apr 25 15:22 18b3fd4d-0860-480a-ae4b-7024710b2f35.ach
-rw------- 1 root root 1304 Apr 25 17:33 191ef2fe-4ef7-4868-99f2-1d5c6e623023.ach
myronjhicks commented 1 year ago

This was exactly our issue during the AWS outage a few days ago. We had 300+ entries at cutoff processing and the merged file never was uploaded due to not being able to write the audit file. We were able to recover the merged file on the pod and upload it directly to the SFTP. We then had to emit the 300+ FileUploaded events manually