httpstan fails to give a response for large outputs

This is following on from stan-dev/pystan#396, which ultimately seems to root from an httpstan issue. I'm working on an MWE, but the case I've identified so far seems to happen with a very large dataset. I used curl to interact with httpstan, to isolate whether it was a Pystan or httpstan error.

In the case I've identified, I'm able to compile and sample a large model, which appears to complete successfully according to the operation endpoint. However, when I do the final GET request to retrieve the model results, httpstan sends a header for the response, but then closes the connection without sending any data. There's no visible error on the console running httpstan (which I started using python -m httpstan), but the curl request fails:

$ curl localhost:8080/v1/models/ex6bdm7x/fits/wxft2x3c
curl: (18) transfer closed with 3144360709 bytes remaining to read

and packet sniffing suggests that httpstan is closing the connection:

12  5.048341    127.0.0.1   127.0.0.1   TCP 68  54876 → 8080 [SYN] Seq=0 Win=65535 Len=0 MSS=16344 WS=64 TSval=3703300602 TSecr=0 SACK_PERM
13  5.048735    127.0.0.1   127.0.0.1   TCP 68  8080 → 54876 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=16344 WS=64 TSval=157578390 TSecr=3703300602 SACK_PERM
14  5.048759    127.0.0.1   127.0.0.1   TCP 56  54876 → 8080 [ACK] Seq=1 Ack=1 Win=408256 Len=0 TSval=3703300602 TSecr=157578390
15  5.048771    127.0.0.1   127.0.0.1   TCP 56  [TCP Window Update] 8080 → 54876 [ACK] Seq=1 Ack=1 Win=408256 Len=0 TSval=157578390 TSecr=3703300602
16  5.048832    127.0.0.1   127.0.0.1   HTTP    165 GET /v1/models/ex6bdm7x/fits/wxft2x3c HTTP/1.1 
17  5.048851    127.0.0.1   127.0.0.1   TCP 56  8080 → 54876 [ACK] Seq=1 Ack=110 Win=408128 Len=0 TSval=157578390 TSecr=3703300602
34  17.511368   127.0.0.1   127.0.0.1   TCP 216 8080 → 54876 [PSH, ACK] Seq=1 Ack=110 Win=408128 Len=160 TSval=157590864 TSecr=3703300602 [TCP segment of a reassembled PDU]
35  17.511463   127.0.0.1   127.0.0.1   TCP 56  54876 → 8080 [ACK] Seq=110 Ack=161 Win=408128 Len=0 TSval=3703313076 TSecr=157590864
36  17.513067   127.0.0.1   127.0.0.1   HTTP    56  HTTP/1.1 200 OK 
37  17.513113   127.0.0.1   127.0.0.1   TCP 56  54876 → 8080 [ACK] Seq=110 Ack=162 Win=408128 Len=0 TSval=3703313077 TSecr=157590865
38  17.513293   127.0.0.1   127.0.0.1   TCP 56  54876 → 8080 [FIN, ACK] Seq=110 Ack=162 Win=408128 Len=0 TSval=3703313077 TSecr=157590865
39  17.513350   127.0.0.1   127.0.0.1   TCP 56  8080 → 54876 [ACK] Seq=162 Ack=111 Win=408128 Len=0 TSval=157590865 TSecr=3703313077

Packet 36 has [FIN, ACK] set.

The corresponding TCP stream is

GET /v1/models/ex6bdm7x/fits/wxft2x3c HTTP/1.1
Host: localhost:8080
User-Agent: curl/8.1.2
Accept: */*

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 3144360709
Date: Wed, 06 Dec 2023 16:26:16 GMT
Server: Python/3.10 aiohttp/3.8.4

stan-dev / httpstan

httpstan fails to give a response for large outputs #652