GitHub API socket hang up

I got this error:

Debug: Populating scores for 2019-07-01
Successfully populated 548 scores
Error: socket hang up (ECONNRESET)
    at ClientRequest.<anonymous> (/home/bmaupin/Dropbox/workspace/personal/langtrends-data/src/GitHub.ts:161:16)
    at ClientRequest.emit (node:events:518:28)
    at ClientRequest.emit (node:domain:488:12)
    at TLSSocket.socketOnEnd (node:_http_client:519:9)
    at TLSSocket.emit (node:events:530:35)
    at TLSSocket.emit (node:domain:488:12)
    at endReadableNT (node:internal/streams/readable:1696:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

Could it be related to https://github.com/nodejs/node/issues/47130 ?

According to that issue, it's introduced in Node 20, which sets keep alive by default for HTTPS requests. The underlying cause is an issue with the HTTP/1.1 spec itself, so can't be fixed.

Seems like possible workarounds are:

Use another HTTP client that doesn't have this issue, such as https://github.com/nodejs/undici

Disable keepalive, e.g.

const agent = new https.Agent({ keepAlive: false });

const options = {
headers: {
  Authorization: `bearer ${this.apiKey}`,
  'User-Agent': 'node.js',
},
hostname: optionsUrl.hostname,
method: 'POST',
path: optionsUrl.pathname,
agent: agent,  // Use custom agent
};

Or add retry logic

  request.on('error', (err: NodeJS.ErrnoException) => {
    if (err.code === 'ECONNRESET') {
      // do something with retryOnError?

This happened when updating all scores after adding new languages, which results in a lot of API calls. So in theory this would be less likely to happen just for the normal monthly scores update.

To do:

[x] Re-run score update after adding new languages to see if the problem occurs again

bmaupin / langtrends-data

GitHub API socket hang up #25