axmb
commented
3 years ago I've loaded #962 into dev, but it's tough to say whether it had an impact. I'm still seeing hang ups after 1.1min -- specifically, the connection appears to be closed on the data service mid-batchValidate request from the curator service.
Error: socket hang up
at connResetException (internal/errors.js:612:14)
at Socket.socketOnEnd (_http_client.js:493:23)
at Socket.emit (events.js:326:22)
at Socket.EventEmitter.emit (domain.js:486:12)
at endReadableNT (_stream_readable.js:1244:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
code: 'ECONNRESET',
config: {
url: 'http://data-dev/api/cases/batchValidate',
method: 'post',
data: '{"cases":[{
[...]
headers: {
Accept: 'application/json, text/plain, */*',
'Content-Type': 'application/json;charset=utf-8',
'User-Agent': 'axios/0.19.2',
'Content-Length': 6478905
},
transformRequest: [ [Function: transformRequest] ],
transformResponse: [ [Function: transformResponse] ],
timeout: 0,
adapter: [Function: httpAdapter],
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: Infinity,
validateStatus: [Function: validateStatus]
},
request: <ref *1> Writable {
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: false,
ending: false,
ended: false,
finished: false,
destroyed: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
afterWriteTickInfo: null,
buffered: [],
bufferedIndex: 0,
allBuffers: true,
allNoop: true,
pendingcb: 0,
prefinished: false,
errorEmitted: false,
emitClose: true,
autoDestroy: true,
errored: false,
closed: false
},
_events: [Object: null prototype] {
response: [Function: handleResponse],
error: [Function: handleRequestError]
},
_eventsCount: 2,
_maxListeners: undefined,
_options: {
protocol: 'http:',
maxRedirects: 21,
maxBodyLength: Infinity,
path: '/api/cases/batchValidate',
method: 'POST',
headers: [Object],
agent: undefined,
agents: [Object],
auth: undefined,
hostname: 'data-dev',
port: null,
nativeProtocols: [Object],
pathname: '/api/cases/batchValidate'
},
_redirectCount: 0,
_redirects: [],
_requestBodyLength: 6478905,
_requestBodyBuffers: [ [Object] ],
_onNativeResponse: [Function (anonymous)],
_currentRequest: ClientRequest {
_events: [Object: null prototype],
_eventsCount: 6,
_maxListeners: undefined,
outputData: [],
outputSize: 0,
writable: true,
destroyed: false,
_last: true,
chunkedEncoding: false,
shouldKeepAlive: false,
useChunkedEncodingByDefault: true,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
_contentLength: null,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
socket: [Socket],
_header: 'POST /api/cases/batchValidate HTTP/1.1\r\n' +
'Accept: application/json, text/plain, */*\r\n' +
'Content-Type: application/json;charset=utf-8\r\n' +
'User-Agent: axios/0.19.2\r\n' +
'Content-Length: 6478905\r\n' +
'Host: data-dev\r\n' +
'Connection: close\r\n' +
'\r\n',
_keepAliveTimeout: 0,
_onPendingData: [Function: noopPendingOutput],
agent: [Agent],
socketPath: undefined,
method: 'POST',
maxHeaderSize: undefined,
insecureHTTPParser: undefined,
path: '/api/cases/batchValidate',
_ended: false,
res: null,
aborted: false,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
reusedSocket: false,
host: 'data-dev',
protocol: 'http:',
_redirectable: [Circular *1],
[Symbol(kCapture)]: false,
[Symbol(kNeedDrain)]: false,
[Symbol(corked)]: 0,
[Symbol(kOutHeaders)]: [Object: null prototype]
},
_currentUrl: 'http://data-dev/api/cases/batchValidate',
[Symbol(kCapture)]: false
},
response: undefined,
isAxiosError: true,
toJSON: [Function (anonymous)]
}Not clear why we'd be terminating the connection on the data service side; I'll take a closer look.
attwad
Describe the bug Requests in dev/prod time out after 60 seconds.
To Reproduce Send an API request, say to batch upsert, with a large amount of data (>10k rows).
Expected behavior Requests, at least batch upsert, should allow more time to complete.
Environment (please complete the following information): Only occurs in dev/prod -- can't repro locally. I've seen the issue both for bulk upload and ADI, so the issue isn't specific to the UI/browser.
I've previously tweaked our nginx config to avoid 504s, but in these cases, the API client gets
500: socket hang up.