Express and middleware that manipulate reponses

[olalonde]

AFAIK, there is currently no generic / documented way to manipulate responses before they are passed on to the underlying socket (through a transform stream for example). I can think of 2 instances where this would be useful: compression and monitoring outgoing bandwidth.

The https://github.com/expressjs/compression/ module resorts to monkey patching the res object which is prone to error, difficult to maintain and not extensible.

It would be nice if there was a documented and extensible way to manipulate responses before they are passed to the underlying socket.

[dougwilson]

Hi! Because all we do is provide a thin app layer oj top of the Node.js core HTTP, without some core-level solution, there isn't too much we can do. If you would like to research, prototype, and share a proposed solution with us, that would be the best way to move this forward :)

[olalonde]

Ok, I will see if I can come up with something (I think I have some idea on how to do it).

[olalonde]

Related issues: https://github.com/strongloop/express/issues/2255 https://github.com/expressjs/compression/issues/13

[olalonde]

WIP:

class ResProxy extends stream.Writable {
  constructor(res) {
    super();
    this.res = res;
    this.origWrite = res.write;
    this.origEnd = res.end;
  }

  write(...args) {
    // console.log('res.write');
    return this.origWrite.apply(this.res, args);
  }

  end(...args) {
    // console.log('res.end');
    return this.origEnd.apply(this.res, args);
  }
}

const patch = (res) => {
  const resProxy = new ResProxy(res);

  res.transforms = [ new stream.PassThrough() ];
  const first = () => {
    return res.transforms[0];
  };
  const last = () => {
    return res.transforms[res.transforms.length - 1];
  };

  res.use = (s) => {
    last().pipe(s);
    res.transforms.push(s);
  };

  let streamConnected = false;
  const connectPipes = () => {
    if (streamConnected) return;
    streamConnected = true;
    last().pipe(resProxy);
  };
  res.write = (...args) => {
    connectPipes();
    // console.log('passthrough res.write');
    return first().write(...args);
  };
  res.end = (...args) => {
    connectPipes();
    // console.log('passthrough res.end');
    return first().end(...args);
  };
};

Usage:

app.use((req, res, next) => {
  patch(res);
  res.use(someTransformStream);
  res.use(someOtherTransformStream);
  res.send('yay!');
});

I have been using the code above for my bandwidth monitoring middleware and initial tests seem positive.

[olalonde]

Created a repo to track progress here: https://github.com/olalonde/express-transform

[chenzx]

I have similar requirements, and i've found express middleware framework doesn't support full-spec http interceptor concepts, like modifying http response.

To be more clear, a middleware, or a http interceptor, should be applied FILO: if it is the first to do with the req, then it will be the last to do with the req. (But i guess this does not occupy with the underlying Node async io mechanism?)

But the above patch code really shocks me: JavaScript code has since evoled a lot! :-)