Node 15 / req.headers not defined

[simllll]

I'm not 100% sure, but I think it is relate dto node15.

There are a lot of cases where req.headers is assumed that is has a value / object, but it's just undefined.

e.g. var ranges = req.headers.range; fails if headers is not defined.

or req.headers['if-match'].... etc

Possible fix: check if req.headers is defined, whenever it is accessed.

Another workaround is ensuring that req.headers is set:

diff --git a/node_modules/send/index.js b/node_modules/send/index.js
index fca2112..782f0a4 100644
--- a/node_modules/send/index.js
+++ b/node_modules/send/index.js
@@ -609,7 +609,12 @@ SendStream.prototype.send = function send (path, stat) {
   var opts = {}
   var res = this.res
   var req = this.req
-  var ranges = req.headers.range
+
+   if (!req.headers) {
+       req.headers = {}
+   }
+
+   var ranges = req.headers.range
   var offset = options.start || 0

   if (headersSent(res)) {

[dougwilson]

Hi @simllll thanks for the repo. I think there is some issue somewhere in your app, as in Node.js 15 (and all other versions) req.headers is always defined as an object. Without req.headers this module cannot work, so if there is no req.headers on your incoming request object, then just don't even pass it here.

I can help investigate if you can provide a test-case that uses the Node.js HTTP server object and shows that req.headers is actually not defined, so we can understand the conditions in which it is undefined.

For reference, here is the Node.js documentation: https://nodejs.org/dist/latest-v15.x/docs/api/http.html#http_message_headers

[simllll]

Interesting, I actually just use a nuxt.js app and had node 15 running.

Here is the "req" object:

req {                                                                                                                                                                                                                  03:04:10
  _readableState: ReadableState {
    objectMode: false,
    highWaterMark: 16384,
    buffer: BufferList {
      head: null,
      tail: null,
      length: 0
    },
    length: 0,
    pipes: [],
    flowing: null,
    ended: true,
    endEmitted: false,
    reading: false,
    constructed: true,
    sync: true,
    needReadable: false,
    emittedReadable: false,
    readableListening: false,
    resumeScheduled: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    destroyed: false,
    errored: null,
    closed: false,
    closeEmitted: false,
    defaultEncoding: 'utf8',
    awaitDrainWriters: null,
    multiAwaitDrain: false,
    readingMore: true,
    decoder: null,
    encoding: null,
    [Symbol(kPaused)]: null
  },
  _events: [Object: null prototype] {
    end: [Function: clearRequestTimeout]
  },
  _eventsCount: 1,
  _maxListeners: undefined,
  socket: <ref *1> Socket {
    connecting: false,
    _hadError: false,
    _parent: null,
    _host: null,
    _readableState: ReadableState {
      objectMode: false,
      highWaterMark: 16384,
      buffer: BufferList {
        head: null,
        tail: null,
        length: 0
      },
      length: 0,
      pipes: [],
      flowing: true,
      ended: false,
      endEmitted: false,
      reading: true,
      constructed: true,
      sync: false,
      needReadable: true,
      emittedReadable: false,
      readableListening: false,
      resumeScheduled: false,
      errorEmitted: false,
      emitClose: false,
      autoDestroy: true,
      destroyed: false,
      errored: null,
      closed: false,
      closeEmitted: false,
      defaultEncoding: 'utf8',
      awaitDrainWriters: null,
      multiAwaitDrain: false,
      readingMore: false,
      decoder: null,
      encoding: null,
      [Symbol(kPaused)]: false
    },
    _events: [Object: null prototype] {
      end: [Array],
      timeout: [Function: socketOnTimeout],
      data: [Function: bound socketOnData],
      error: [Function: socketOnError],
      close: [Array],
      drain: [Function: bound socketOnDrain],
      resume: [Function: onSocketResume],
      pause: [Function: onSocketPause]
    },
    _eventsCount: 8,
    _maxListeners: undefined,
    _writableState: WritableState {
      objectMode: false,
      highWaterMark: 16384,
      finalCalled: false,
      needDrain: false,
      ending: false,
      ended: false,
      finished: false,
      destroyed: false,
      decodeStrings: false,
      defaultEncoding: 'utf8',
      length: 0,
      writing: false,
      corked: 0,
      sync: true,
      bufferProcessing: false,
      onwrite: [Function: bound onwrite],
      writecb: null,
      writelen: 0,
      afterWriteTickInfo: null,
      buffered: [],
      bufferedIndex: 0,
      allBuffers: true,
      allNoop: true,
      pendingcb: 0,
      constructed: true,
      prefinished: false,
      errorEmitted: false,
      emitClose: false,
      autoDestroy: true,
      errored: null,
      closed: false,
      closeEmitted: false,
      [Symbol(kOnFinished)]: []
    },
    allowHalfOpen: true,
    _sockname: null,
    _pendingData: null,
    _pendingEncoding: '',
    server: Server {
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      _events: [Object: null prototype],
      _eventsCount: 3,
      _maxListeners: undefined,
      _connections: 7,
      _handle: [TCP],
      _usingWorkers: false,
      _workers: [],
      _unref: false,
      allowHalfOpen: true,
      pauseOnConnect: false,
      httpAllowHalfOpen: false,
      timeout: 0,
      keepAliveTimeout: 5000,
      maxHeadersCount: null,
      headersTimeout: 60000,
      requestTimeout: 0,
      _connectionKey: '4:0.0.0.0:3001',
      destroy: [Function (anonymous)],
      [Symbol(IncomingMessage)]: [Function: IncomingMessage],
      [Symbol(ServerResponse)]: [Function: ServerResponse],
      [Symbol(kCapture)]: false,
      [Symbol(async_id_symbol)]: 53
    },
    _server: Server {
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      _events: [Object: null prototype],
      _eventsCount: 3,
      _maxListeners: undefined,
      _connections: 7,
      _handle: [TCP],
      _usingWorkers: false,
      _workers: [],
      _unref: false,
      allowHalfOpen: true,
      pauseOnConnect: false,
      httpAllowHalfOpen: false,
      timeout: 0,
      keepAliveTimeout: 5000,
      maxHeadersCount: null,
      headersTimeout: 60000,
      requestTimeout: 0,
      _connectionKey: '4:0.0.0.0:3001',
      destroy: [Function (anonymous)],
      [Symbol(IncomingMessage)]: [Function: IncomingMessage],
      [Symbol(ServerResponse)]: [Function: ServerResponse],
      [Symbol(kCapture)]: false,
      [Symbol(async_id_symbol)]: 53
    },
    parser: HTTPParser {
      '0': [Function: bound setRequestTimeout],
      '1': [Function: parserOnHeaders],
      '2': [Function: parserOnHeadersComplete],
      '3': [Function: parserOnBody],
      '4': [Function: parserOnMessageComplete],
      '5': [Function: bound onParserExecute],
      '6': [Function: bound onParserTimeout],
      _headers: [],
      _url: '',
      socket: [Circular *1],
      incoming: [IncomingMessage],
      outgoing: null,
      maxHeaderPairs: 2000,
      _consumed: true,
      onIncoming: [Function: bound parserOnIncoming],
      [Symbol(resource_symbol)]: [HTTPServerAsyncResource]
    },
    on: [Function: socketListenerWrap],
    addListener: [Function: socketListenerWrap],
    prependListener: [Function: socketListenerWrap],
    setEncoding: [Function: socketSetEncoding],
    _paused: false,
    _peername: {
      address: '127.0.0.1',
      family: 'IPv4',
      port: 59460
    },
    _httpMessage: ServerResponse {
      _events: [Object: null prototype],
      _eventsCount: 1,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: false,
      _last: false,
      chunkedEncoding: false,
      shouldKeepAlive: true,
      _defaultKeepAlive: true,
      useChunkedEncodingByDefault: true,
      sendDate: true,
      _removedConnection: false,
      _removedContLen: false,
      _removedTE: false,
      _contentLength: null,
      _hasBody: true,
      _trailer: '',
      finished: false,
      _headerSent: false,
      _closed: false,
      socket: [Circular *1],
      _header: null,
      _keepAliveTimeout: 5000,
      _onPendingData: [Function: bound updateOutgoingData],
      _sent100: false,
      _expect_continue: false,
      locals: {},
      [Symbol(kCapture)]: false,
      [Symbol(kNeedDrain)]: false,
      [Symbol(corked)]: 0,
      [Symbol(kOutHeaders)]: [Object: null prototype]
    },
    [Symbol(async_id_symbol)]: 1039470,
    [Symbol(kHandle)]: TCP {
      reading: true,
      onconnection: null,
      _consumed: true,
      [Symbol(owner_symbol)]: [Circular *1]
    },
    [Symbol(kSetNoDelay)]: false,
    [Symbol(lastWriteQueueSize)]: 0,
    [Symbol(timeout)]: null,
    [Symbol(kBuffer)]: null,
    [Symbol(kBufferCb)]: null,
    [Symbol(kBufferGen)]: null,
    [Symbol(kCapture)]: false,
    [Symbol(kBytesRead)]: 0,
    [Symbol(kBytesWritten)]: 0,
    [Symbol(RequestTimeout)]: undefined
  },
  httpVersionMajor: 1,
  httpVersionMinor: 1,
  httpVersion: '1.1',
  complete: true,
  rawHeaders: [
    'Host',
    'localhost:3001',
    'Connection',
    'keep-alive',
    'Pragma',
    'no-cache',
    'Cache-Control',
    'no-cache',
    'User-Agent',
    'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36',
    'DNT',
    '1',
    'Accept',
    '*/*',
    'Sec-Fetch-Site',
    'same-origin',
    'Sec-Fetch-Mode',
    'cors',
    'Sec-Fetch-Dest',
    'empty',
    'Referer',
    'http://localhost:3001/pwa/company/dashboard',
    'Accept-Encoding',
    'gzip, deflate, br',
    'Accept-Language',
    'de-AT,de;q=0.9,en-AT;q=0.8,en;q=0.7,de-DE;q=0.6,en-US;q=0.5'
  ],
  rawTrailers: [],
  aborted: false,
  upgrade: false,
  url: '/manifest.3cf9e431.json',
  method: 'GET',
  statusCode: null,
  statusMessage: null,
  client: <ref *1> Socket {
    connecting: false,
    _hadError: false,
    _parent: null,
    _host: null,
    _readableState: ReadableState {
      objectMode: false,
      highWaterMark: 16384,
      buffer: BufferList {
        head: null,
        tail: null,
        length: 0
      },
      length: 0,
      pipes: [],
      flowing: true,
      ended: false,
      endEmitted: false,
      reading: true,
      constructed: true,
      sync: false,
      needReadable: true,
      emittedReadable: false,
      readableListening: false,
      resumeScheduled: false,
      errorEmitted: false,
      emitClose: false,
      autoDestroy: true,
      destroyed: false,
      errored: null,
      closed: false,
      closeEmitted: false,
      defaultEncoding: 'utf8',
      awaitDrainWriters: null,
      multiAwaitDrain: false,
      readingMore: false,
      decoder: null,
      encoding: null,
      [Symbol(kPaused)]: false
    },
    _events: [Object: null prototype] {
      end: [Array],
      timeout: [Function: socketOnTimeout],
      data: [Function: bound socketOnData],
      error: [Function: socketOnError],
      close: [Array],
      drain: [Function: bound socketOnDrain],
      resume: [Function: onSocketResume],
      pause: [Function: onSocketPause]
    },
    _eventsCount: 8,
    _maxListeners: undefined,
    _writableState: WritableState {
      objectMode: false,
      highWaterMark: 16384,
      finalCalled: false,
      needDrain: false,
      ending: false,
      ended: false,
      finished: false,
      destroyed: false,
      decodeStrings: false,
      defaultEncoding: 'utf8',
      length: 0,
      writing: false,
      corked: 0,
      sync: true,
      bufferProcessing: false,
      onwrite: [Function: bound onwrite],
      writecb: null,
      writelen: 0,
      afterWriteTickInfo: null,
      buffered: [],
      bufferedIndex: 0,
      allBuffers: true,
      allNoop: true,
      pendingcb: 0,
      constructed: true,
      prefinished: false,
      errorEmitted: false,
      emitClose: false,
      autoDestroy: true,
      errored: null,
      closed: false,
      closeEmitted: false,
      [Symbol(kOnFinished)]: []
    },
    allowHalfOpen: true,
    _sockname: null,
    _pendingData: null,
    _pendingEncoding: '',
    server: Server {
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      _events: [Object: null prototype],
      _eventsCount: 3,
      _maxListeners: undefined,
      _connections: 7,
      _handle: [TCP],
      _usingWorkers: false,
      _workers: [],
      _unref: false,
      allowHalfOpen: true,
      pauseOnConnect: false,
      httpAllowHalfOpen: false,
      timeout: 0,
      keepAliveTimeout: 5000,
      maxHeadersCount: null,
      headersTimeout: 60000,
      requestTimeout: 0,
      _connectionKey: '4:0.0.0.0:3001',
      destroy: [Function (anonymous)],
      [Symbol(IncomingMessage)]: [Function: IncomingMessage],
      [Symbol(ServerResponse)]: [Function: ServerResponse],
      [Symbol(kCapture)]: false,
      [Symbol(async_id_symbol)]: 53
    },
    _server: Server {
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      _events: [Object: null prototype],
      _eventsCount: 3,
      _maxListeners: undefined,
      _connections: 7,
      _handle: [TCP],
      _usingWorkers: false,
      _workers: [],
      _unref: false,
      allowHalfOpen: true,
      pauseOnConnect: false,
      httpAllowHalfOpen: false,
      timeout: 0,
      keepAliveTimeout: 5000,
      maxHeadersCount: null,
      headersTimeout: 60000,
      requestTimeout: 0,
      _connectionKey: '4:0.0.0.0:3001',
      destroy: [Function (anonymous)],
      [Symbol(IncomingMessage)]: [Function: IncomingMessage],
      [Symbol(ServerResponse)]: [Function: ServerResponse],
      [Symbol(kCapture)]: false,
      [Symbol(async_id_symbol)]: 53
    },
    parser: HTTPParser {
      '0': [Function: bound setRequestTimeout],
      '1': [Function: parserOnHeaders],
      '2': [Function: parserOnHeadersComplete],
      '3': [Function: parserOnBody],
      '4': [Function: parserOnMessageComplete],
      '5': [Function: bound onParserExecute],
      '6': [Function: bound onParserTimeout],
      _headers: [],
      _url: '',
      socket: [Circular *1],
      incoming: [IncomingMessage],
      outgoing: null,
      maxHeaderPairs: 2000,
      _consumed: true,
      onIncoming: [Function: bound parserOnIncoming],
      [Symbol(resource_symbol)]: [HTTPServerAsyncResource]
    },
    on: [Function: socketListenerWrap],
    addListener: [Function: socketListenerWrap],
    prependListener: [Function: socketListenerWrap],
    setEncoding: [Function: socketSetEncoding],
    _paused: false,
    _peername: {
      address: '127.0.0.1',
      family: 'IPv4',
      port: 59460
    },
    _httpMessage: ServerResponse {
      _events: [Object: null prototype],
      _eventsCount: 1,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: false,
      _last: false,
      chunkedEncoding: false,
      shouldKeepAlive: true,
      _defaultKeepAlive: true,
      useChunkedEncodingByDefault: true,
      sendDate: true,
      _removedConnection: false,
      _removedContLen: false,
      _removedTE: false,
      _contentLength: null,
      _hasBody: true,
      _trailer: '',
      finished: false,
      _headerSent: false,
      _closed: false,
      socket: [Circular *1],
      _header: null,
      _keepAliveTimeout: 5000,
      _onPendingData: [Function: bound updateOutgoingData],
      _sent100: false,
      _expect_continue: false,
      locals: {},
      [Symbol(kCapture)]: false,
      [Symbol(kNeedDrain)]: false,
      [Symbol(corked)]: 0,
      [Symbol(kOutHeaders)]: [Object: null prototype]
    },
    [Symbol(async_id_symbol)]: 1039470,
    [Symbol(kHandle)]: TCP {
      reading: true,
      onconnection: null,
      _consumed: true,
      [Symbol(owner_symbol)]: [Circular *1]
    },
    [Symbol(kSetNoDelay)]: false,
    [Symbol(lastWriteQueueSize)]: 0,
    [Symbol(timeout)]: null,
    [Symbol(kBuffer)]: null,
    [Symbol(kBufferCb)]: null,
    [Symbol(kBufferGen)]: null,
    [Symbol(kCapture)]: false,
    [Symbol(kBytesRead)]: 0,
    [Symbol(kBytesWritten)]: 0,
    [Symbol(RequestTimeout)]: undefined
  },
  _consuming: false,
  _dumped: false,
  originalUrl: '/manifest.3cf9e431.json',
  _parsedUrl: Url {
    protocol: null,
    slashes: null,
    auth: null,
    host: null,
    port: null,
    hostname: null,
    hash: null,
    search: null,
    query: null,
    pathname: '/manifest.3cf9e431.json',
    path: '/manifest.3cf9e431.json',
    href: '/manifest.3cf9e431.json',
    _raw: '/manifest.3cf9e431.json'
  },
  _parsedOriginalUrl: Url {
    protocol: null,
    slashes: null,
    auth: null,
    host: null,
    port: null,
    hostname: null,
    hash: null,
    search: null,
    query: null,
    pathname: '/manifest.3cf9e431.json',
    path: '/manifest.3cf9e431.json',
    href: '/manifest.3cf9e431.json',
    _raw: '/manifest.3cf9e431.json'
  },
  [Symbol(kCapture)]: false,
  [Symbol(kHeaders)]: {
    host: 'localhost:3001',
    connection: 'keep-alive',
    pragma: 'no-cache',
    'cache-control': 'no-cache',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36',
    dnt: '1',
    accept: '*/*',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    'sec-fetch-dest': 'empty',
    referer: 'http://localhost:3001/pwa/company/dashboard',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'de-AT,de;q=0.9,en-AT;q=0.8,en;q=0.7,de-DE;q=0.6,en-US;q=0.5'
  },
  [Symbol(kHeadersCount)]: 26,
  [Symbol(kTrailers)]: null,
  [Symbol(kTrailersCount)]: 0,
  [Symbol(RequestTimeout)]: undefined
}
req.headers undefined  

Let me check with an older version of node...

[dougwilson]

Hi @simllll how are you actually trying to read req.headers? In a recent 15.x update, the headers property has become a non-enumerable property on the prototype that references that kHeaders symbol, and I assume that log output above is only including enumerable properties, not the non-enumerable properties on the prototype, so it would naturally not appear in that output.

[dougwilson]

If you are going to try other Node.js versions, I would suggest trying out 15.0.0 or 15.0.1. If your issue does not occur on those two versions but it does occur on 5.1.0/5.2.0, this is very likely a Node.js accidental regression somewhere that Node.js would very likely want to fix in the 15.x line.

[simllll]

It's a simple output with console.log('req', req) and console.log('req.headers', req.headers). If I try to output "req.headers.range" it fails already with a typeerror

Quick test results: Node 14.15.0: works Node 15.0.1: works Node 15.1.0: TypeError: Cannot read property 'range' of undefined Node 15.2.0: TypeError: Cannot read property 'range' of undefined

So your assumption looks pretty much right ;-)

Thanks for your quick reply!

[dougwilson]

Awesome, thanks for the testing! You will need to report this to Node.js as a regress for them to correct, as req.headers is part of their API to access incoming request headers.

[dougwilson]

If it helps, I just ran this module's test suite with all the Node.js versions you listed above and it ran without issue, and req.headers was defined in every version. I know you noted you're using nuxt.js, so it may be something more complicated; something not playing well with the change to how `req.headers works in 15.1.0+ in your dependency chain prior to when you invoke this module.

[simllll]

Thanks, I will look into this next week :+1: I will keep you posted.

Thanks again for your work and time ! :)

[simllll]

I think I found the culprit here, but not quite sure how to solve it.

The issue seems the spread operator (makes kinda sense when you said they changed it to a symbol) .

serveAssets(
        {
            ...req,
            originalUrl: req.originalUrl.replace('/website/_assets', '') // remove website/_assets from path
        },
        res,
        next
    );

the quesiton is now, what's the best way to do this now. I would like to have a copy of req with a different originalUrl, but doing {...req} removes the header field. Any hints?

[simllll]

fixes it, but it doesn't feel right. What if they change another property to a symbol,it will also be lost.

[dougwilson]

Reporting this to Node.js as a use case that is broken with the change is likely the best path.