MessageFragmenter doesn't take account of encryption overhead

[ctz]

If we set ServerConfig::max_fragment_size or ClientConfig::max_fragment_size it is expected that all the TLS messages are no larger than this. However, this is only true if the messages are unencrypted: after that the sizes are larger by the encryption overhead.

This means, setting max_fragment_size to Some(64), the message sizes are:

64, 64, 9, 6, 32, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, 81

[jbr]

I'm reading RFC 6066 and it seems like 81 would be a reasonable length at the record layer since that the max_fragment_size limits plaintext length, not the protected length at the record layer.

The negotiated length limits the input that the record layer may process without fragmentation (that is, the maximum value of TLSPlaintext.length; see [RFC5246], Section 6.2.1). Note that the output of the record layer may be larger. For example, if the negotiated length is 2^9=512, then, when using currently defined cipher suites (those defined in [RFC5246] and [RFC2712]) and null compression, the record-layer output can be at most 805 bytes: 5 bytes of headers, 512 bytes of application data, 256 bytes of padding, and 32 bytes of MAC. This means that in this event a TLS record-layer peer receiving a TLS record-layer message larger than 805 bytes MUST discard the message and send a "record_overflow" alert, without decrypting the message. When this extension is used with Datagram Transport Layer Security (DTLS), implementations SHOULD NOT generate record_overflow alerts unless the packet passes message authentication.

If I'm misreading that, would we need to determine the record payload protection overhead before fragmenting in order to fragment shorter?

[ctz]

We don't implement RFC6066 max_fragment_size negotiation (reasons here: https://github.com/rustls/rustls/issues/585#issuecomment-803130069); instead we just have a control for the size of fragments we produce. The goal for that feature is that applications can provide their TCP MSS and always get TLS fragments that fit in one TCP packet.

would we need to determine the record payload protection overhead before fragmenting in order to fragment shorter?

Yes I think so.

[ctz]

1579 is relevant to this