Allow Missing File Header does not appear to work on POST `/files/create` using ACH text body

[emarcey]

ACH Version

1.34.2

What were you trying to do?

I am attempting to handle support for a system where both properly-formatted ACH files, and custom return files that are missing file headers/controls are being processed.

I added the allowMissingFileHeader=true to the request params and attempted to process both types of files

What did you expect to see?

I expected to see a success response for both files, bypassing the missing file header/control in the latter.

What did you see?

Instead, I received 500 errors for both the valid and the invalid files. I certainly expected the valid file to still work correctly, but having the invalid file not work causes issues for me.

Successful response with valid file (all data is mocked)

Error response with invalid file (all data is mocked)

How can we reproduce the problem?

1. Use Moov ACH 1.34.2
2. Use url with params localhost:8080/files/create?bypassDestination=true&bypassOrigin=true&customTraceNumbers=true&bypassCompanyIdentificationMatch=true&allowInvalidAmounts=true&allowMissingFileControl=true&allowMissingFileHeader=true
3. Use attached sample file employers_test.txt

[emarcey]

In version 1.20.0, the bug did not occur, but the flag also didn't work correctly for an invalid file

[adamdecaf]

I'm not able to reproduce the 500 error with your request. Can you verify the container is running and our requests look the same?

PR with test: https://github.com/moov-io/ach/pull/1379

Test Output

``` $ go test ./test/issues/... -count 1 -v -run TestIssue1378 === RUN TestIssue1378 Origin OriginName Destination DestinationName FileCreationDate FileCreationTime N833219776 KRAB PAYROLL SERVICES 011600062 SAMPLE BANK TWO 210518 2246 BatchNumber SECCode ServiceClassCode CompanyName DiscretionaryData Identification EntryDescription EffectiveEntryDate DescriptiveDate 0000001 PPD 200 (Mixed Debits/Credits) Apple 900 *1069242268 1833219778 NET=PAY 240425 240425 TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 07590057 44442280852 444400 Kyle Prince 011600060000001 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 07590057 44442280852 12345 Michael Moyer 011600060000002 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 07590057 44442280852 434354 Tracy Delgado 011600060000003 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 27 (Checking Debit) 01140153 1111222233330000 891099 Apple 011600060000004 Forward ServiceClassCode EntryAddendaCount EntryHash TotalDebits TotalCredits MACCode ODFIIdentification BatchNumber 200 (Mixed Debits/Credits) 000004 0023910324 891099 891099 01160006 0000001 BatchNumber SECCode ServiceClassCode CompanyName DiscretionaryData Identification EntryDescription EffectiveEntryDate DescriptiveDate 0000002 PPD 200 (Mixed Debits/Credits) Key-Morse 900 *1069242268 1833219778 NET=PAY 240425 240425 TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 07590057 44442280852 2094 Laura Cruz 011600060000005 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 05300021 44449593224 1604 Jillian Clayton 011600060000006 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 05300021 44449593224 10766499 Brian Waters 011600060000007 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 27 (Checking Debit) 01804265 121889252 10770197 Key-Morse 011600060000008 Forward ServiceClassCode EntryAddendaCount EntryHash TotalDebits TotalCredits MACCode ODFIIdentification BatchNumber 200 (Mixed Debits/Credits) 000004 0019994364 10770197 10770197 01160006 0000002 BatchNumber SECCode ServiceClassCode CompanyName DiscretionaryData Identification EntryDescription EffectiveEntryDate DescriptiveDate 0000003 PPD 200 (Mixed Debits/Credits) Rubio Baldwin 900 *1069242268 1833219778 NET=PAY 240425 240425 TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 07100001 4444911717 8617 Stephanie Flowers 011600060000009 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 07100001 4444911717 6483 Catherine Martin 011600060000010 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 07100001 4444911717 4566 Jason Lewis 011600060000011 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 27 (Checking Debit) 04210119 44444383427 19666 Rubio Baldwin 011600060000012 Forward ServiceClassCode EntryAddendaCount EntryHash TotalDebits TotalCredits MACCode ODFIIdentification BatchNumber 200 (Mixed Debits/Credits) 000004 0025510122 19666 19666 01160006 0000003 BatchNumber SECCode ServiceClassCode CompanyName DiscretionaryData Identification EntryDescription EffectiveEntryDate DescriptiveDate 0000004 PPD 200 (Mixed Debits/Credits) PRENOTES 900 *1069242268 1833219778 NET=PAY 240425 240425 TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 23 (Checking Prenote Credit) 02100002 986171749 0 Blake Webb 011600060000013 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 23 (Checking Prenote Credit) 02100002 986171748 0 Kayla Rodriguez 011600060000014 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 33 (Savings Prenote Credit) 02100002 011600060000015 0 Julie Decker 011600060000015 Forward ServiceClassCode EntryAddendaCount EntryHash TotalDebits TotalCredits MACCode ODFIIdentification BatchNumber 200 (Mixed Debits/Credits) 000003 0006300006 0 0 01160006 0000004 BatchNumber SECCode ServiceClassCode CompanyName DiscretionaryData Identification EntryDescription EffectiveEntryDate DescriptiveDate 0000005 PPD 200 (Mixed Debits/Credits) Lopez-Robinson 900 *1069242268 1833219778 NET=PAY 240425 240425 TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 12430312 4444379179753 70690 Robin Fleming 011600060000016 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 22 (Checking Credit) 12430312 444459015 955239 Christine Keith 011600060000017 Forward TransactionCode RDFIIdentification AccountNumber Amount Name TraceNumber Category 27 (Checking Debit) 32117526 4444389 1025929 Lopez-Robinson 011600060000018 Forward ServiceClassCode EntryAddendaCount EntryHash TotalDebits TotalCredits MACCode ODFIIdentification BatchNumber 200 (Mixed Debits/Credits) 000003 0056978150 1025929 1025929 01160006 0000005 BatchCount BlockCount EntryAddendaCount TotalDebitAmount TotalCreditAmount 000005 000003 00000018 12706891 12706891 --- PASS: TestIssue1378 (0.00s) PASS ok github.com/moov-io/ach/test/issues 1.056s ```

[emarcey]

It does also appear to work fine when I copy your test code and run that

Here's my docker ps:

I copy pasted the exact url into my postman and hit the docker image, with the same result as before:

[adamdecaf]

Can you dump the full request headers and the moov/ach logs?

[emarcey]

Here's the curl export:

--header 'Content-Type: text/plain' \
--data '101 011600062N8332197762105182246A094101SAMPLE BANK TWO        KRAB PAYROLL SERVICES  00000000
5200Apple           900      *10692422681833219778PPDNET=PAY   240425240425   1011600060000001
62207590057544442280852      0000444400011600060000001Kyle Prince             0011600060000001
62207590057544442280852      0000012345011600060000002Michael Moyer           0011600060000002
62207590057544442280852      0000434354011600060000003Tracy Delgado           0011600060000003
6270114015331111222233330000 0000891099011600060000004Apple                   0011600060000004
820000000400239103240000008910990000008910991833219778                         011600060000001
5200Key-Morse       900      *10692422681833219778PPDNET=PAY   240425240425   1011600060000002
62207590057544442280852      0000002094011600060000005Laura Cruz              0011600060000005
62205300021944449593224      0000001604011600060000006Jillian Clayton         0011600060000006
62205300021944449593224      0010766499011600060000007Brian Waters            0011600060000007
627018042652121889252        0010770197011600068000008Key-Morse               0011600060000008
820000000400199943640000107701970000107701971833219778                         011600060000002
5200Rubio Baldwin   900      *10692422681833219778PPDNET=PAY   240425240425   1011600060000003
6220710000134444911717       0000008617011600060000009Stephanie Flowers       0011600060000009
6220710000134444911717       0000006483011600060000010Catherine Martin        0011600060000010
6220710000134444911717       0000004566011600060000011Jason Lewis             0011600060000011
62704210119044444383427      0000019666011600068000012Rubio Baldwin           0011600060000012
820000000400255101220000000196660000000196661833219778                         011600060000003
5200PRENOTES        900      *10692422681833219778PPDNET=PAY   240425240425   1011600060000004
623021000021986171749        0000000000011600060000013Blake Webb              0011600060000013
623021000021986171748        0000000000011600060000014Kayla Rodriguez         0011600060000014
633021000021011600060000015  0000000000287            Julie Decker            0011600060000015
820000000300063000060000000000000000000000001833219778                         011600060000004
5200Lopez-Robinson  900      *10692422681833219778PPDNET=PAY   240425240425   1011600060000005
6221243031204444379179753    0000070690011600060000016Robin Fleming           0011600060000016
622124303120444459015        0000955239011600060000017Christine Keith         0011600060000017
6273211752614444389          0001025929011600068000018Lopez-Robinson          0011600060000018
820000000300569781500000010259290000010259291833219778                         011600060000005
9000005000003000000180132692966000012706891000012706891
'

I don't have much in the way of ACH logs

Is there a debug enabled setting I can use?

For what it's worth, I built ./bin/server locally and that seems to work fine as well (although allowMissingFileHeader/Control don't work to allow invalid files through, but at least the bug doesn't show up). It's just the docker image where I'm running into issues

[emarcey]

It also works okay if I submit a JSON file for creation without a file header/control, it's only on the ACH formatted version where there's an issue, so I would assume the problem is at these two places: https://github.com/moov-io/ach/blob/master/reader.go#L205 https://github.com/moov-io/ach/blob/master/reader.go#L211

[emarcey]

Okay so idk why but v1.34.3 that you just released seems to be working for me with properly formatted files.

I'm unable to create with an ach file when it's missing the file header/control, even when using that opt. Is that just not supported?

[adamdecaf]

Trying out that curl works for me. I had to add some args to the curl command.

curl -v -XPOST http://localhost:8080/files/create --header 'Content-Type: text/plain' \

*   Trying [::1]:8080...
* Connected to localhost (::1) port 8080
> POST /files/create HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.4.0
> Accept: */*
> Content-Type: text/plain
> Content-Length: 2811
> 
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Date: Mon, 26 Feb 2024 18:14:25 GMT
< Transfer-Encoding: chunked
< 
{"id":"1ca6541d6eb6a363ac2f4a371b5fbe9d07ab9e79","file":{"id":"1ca6541d6eb6a363ac2f4a371b5fbe9d07ab9e79","fileHeader":{"id":"","immediateDestination":"011600062","immediateOrigin":"N833219776","fileCreationDate":"210518","fileCreationTime":"2246","fileIDModifier":"A","immediateDestinationName":"SAMPLE BANK TWO","immediateOriginName":"KRAB PAYROLL SERVICES","referenceCode":"00000000"},

[adamdecaf]

I see the issue. Without a header I'm unable to get a file created, even with those query parameters. I'll open a fix.

[emarcey]

Thanks for the quick response @adamdecaf !

[adamdecaf]

I've updated https://github.com/moov-io/ach/pull/1379 with a fix. Surprised this has lasted so long.

[emarcey]

Awesome, thank you so much! I'll keep an eye out for a new release

[adamdecaf]

Released in https://github.com/moov-io/ach/releases/tag/v1.34.4

[emarcey]

It's working for me, thanks!