HTTPS Requests are Failing with Node 9

[tonymet]

It seems like there's a regression with HTTPS urls in Node 9 that's not there in node8. Here's an example test run

Steps to Repeat

$ docker run  -it -v$(pwd):/app node:9-alpine /bin/sh
# inside container
$ NODE_DEBUG=net,http,https yarn test -- --grep "matching an https"

Actual Results

  Replay
    matching an https url
      1) "before all" hook

  0 passing (5s)
  1 failing

  1) Replay
       matching an https url
         "before all" hook:
     Error: Timeout of 5000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

Expected Results

the "matching https url" should pass, which would indicate working support for HTTPS requests.

Suspicions

• debug output (see below) shows 2 separate requests
• debug logs for https proxy server are not present -- maybe node is blocking
• could node9 be preventing the dns-patcher from working?

Full Debug Output

/app # NODE_DEBUG=http,https,net,tls DEBUG=replay  yarn test -- --grep 'matching an https'
NET 710: _read
NET 710: Socket._read readStart
yarn run v1.3.2
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
$ mocha --grep 'matching an https'

  Replay
    matching an https url
HTTP 737: call onSocket 0 0
HTTP 737: createConnection example.com:3443:::::::false:: { servername: 'example.com',
  _defaultAgent:
   Agent {
     domain: null,
     _events: { free: [Function] },
     _eventsCount: 1,
     _maxListeners: undefined,
     defaultPort: 443,
     protocol: 'https:',
     options: { path: null },
     requests: {},
     sockets: { 'example.com:3443:::::::false::': [] },
     freeSockets: {},
     keepAliveMsecs: 1000,
     keepAlive: false,
     maxSockets: Infinity,
     maxFreeSockets: 256,
     maxCachedSessions: 100,
     _sessionCache: { map: {}, list: [] } },
  timeout: 500,
  hostname: 'example.com',
  port: 3443,
  path: null,
  rejectUnauthorized: false,
  host: 'example.com',
  _agentKey: 'example.com:3443:::::::false::' }
HTTPS 737: createConnection { servername: 'example.com',
  _defaultAgent:
   Agent {
     domain: null,
     _events: { free: [Function] },
     _eventsCount: 1,
     _maxListeners: undefined,
     defaultPort: 443,
     protocol: 'https:',
     options: { path: null },
     requests: {},
     sockets: { 'example.com:3443:::::::false::': [] },
     freeSockets: {},
     keepAliveMsecs: 1000,
     keepAlive: false,
     maxSockets: Infinity,
     maxFreeSockets: 256,
     maxCachedSessions: 100,
     _sessionCache: { map: {}, list: [] } },
  timeout: 500,
  hostname: 'example.com',
  port: 3443,
  path: null,
  rejectUnauthorized: false,
  host: 'example.com',
  _agentKey: 'example.com:3443:::::::false::',
  encoding: null }
NET 737: pipe false null
NET 737: connect: find host example.com
NET 737: connect: dns options { family: undefined, hints: 32 }
HTTP 737: sockets example.com:3443:::::::false:: 1
HTTP 737: outgoing message end.
NET 737: _read
NET 737: _read wait for connection
      1) "before all" hook

Build Info

git log -1|cat
68cc2bc - (HEAD -> master, origin/master, origin/HEAD) Merge pull request #140 from albertogasparin/fix/ignore-dot-files (28 hours ago) <Assaf Arkin>%
 node -v
v9.2.1
/ # uname -a
Linux 10ee846c2cf3 4.9.49-moby #1 SMP Wed Sep 27 23:17:17 UTC 2017 x86_64 Linux

[g-sam]

Any plans to fix this?

[assaf]

Test case?

[g-sam]

We are using replay to mock calls to a https endpoint.

On Node 9: DEBUG=replay gives no output and the real endpoint is hit. On Node 8: it works as expected.

Let me know if you need something more comprehensive but @tonymet has already given steps to reproduce above.

[ostrgard]

The issue is reproduceable following the steps in this repo: https://github.com/ostrgard/node-replay-bug. Node's http and https seems to work fine, but other fetching libraries have issues with https requests.

Adding HTTPS.request = HTTP.request; to the end of src/patch_http_request.js fixes this, but does fail some tests.

Hope this is of some help, @assaf.

[nrebhun]

I may be experiencing a similar issue with Node 10.6.0

I don't know when I'll have time to add test cases/steps to reproduce (I will try) but I wanted to put this on someone's radar. Maybe someone will beat me to it, and help solve this thing.

[Sushmeet]

Hi Was wondering if there has been any progress made on this. We love using the library but ever seen moving to node 10, we are unable to add new tests. Any short term solution would be accepted too. We tried @ostrgard. solution but it didn't work out for us. Thanks in advance.

FYI , We have been able to get some progress by applying the suggestion here for supertest. https://github.com/assaf/node-replay/issues/147

[hex0cter]

+1

[dsagal]

For the sake of others who end up on this thread, at least in my case, replay works fine with axios on Node 10. My problem was that replay was ignoring requests to localhost. (Solved by calling Replay.reset('localhost').)

It's hard to tell what's wrong if replay doesn't record requests. It would be helpful if it would log a debug message for skipped requests.