RESOURCE_EXHAUSTED: Received message larger than max (808722544 vs 4194304) in version 1.10.9

[0x01F4]

Problem description

After upgrading to version 1.10.9(nodejs grpc client), started getting RESOURCE_EXHAUSTED: Received message larger than max (808722544 vs 4194304) randomly.

[0x01F4]

@murgatroid99 Maybe this related to https://github.com/grpc/grpc-node/commit/674f4e351a619fd4532f84ae6dff96b8ee4e1ed3?

Maybe size counter is not reset after each stream?

[0x01F4]

It is happening when there is traefik in between client and server and traefik return 404.

This is not happening before https://github.com/grpc/grpc-node/commit/674f4e351a619fd4532f84ae6dff96b8ee4e1ed3

Wireshark capture:

Frame 2393: 131 bytes on wire (1048 bits), 131 bytes captured (1048 bits) on interface en0, id 0 Ethernet II, Src: Apple_67:50:66 (f4:d4:88:67:50:66), Dst: Apple_55:ee:e0 (88:66:5a:55:ee:e0) Internet Protocol Version 4, Src: 10.8.20.228, Dst: 10.8.20.115 Transmission Control Protocol, Src Port: 8046, Dst Port: 57478, Seq: 1, Ack: 702, Len: 65 HyperText Transfer Protocol 2 HyperText Transfer Protocol 2 GRPC Malformed message data: only 19 bytes left, need at least 808722549 bytes. [Expert Info (Warning/Undecoded): GRPC Malformed message data: only 19 bytes left, need at least 808722549 bytes.] [GRPC Malformed message data: only 19 bytes left, need at least 808722549 bytes.] [Severity level: Warning] [Group: Undecoded]

[0x01F4]

Closing the issue.Adding details incase someone stumble upon the same.

This is happening since traefik is returning non-grpc response and grpc-node tries to parse first four bytes as length of the message in this case which "404 " in ASCII (52,48,52,32).

function readUInt32BE(offset = 0) {
  validateNumber(offset, 'offset');
  const first = this[offset];
  const last = this[offset + 3];
  if (first === undefined || last === undefined)
    boundsError(offset, this.length - 4);

  return first * 2 ** 24 +
    this[++offset] * 2 ** 16 +
    this[++offset] * 2 ** 8 +
    last;
}

It was working before https://github.com/grpc/grpc-node/commit/674f4e351a619fd4532f84ae6dff96b8ee4e1ed3 since this check was not there.