docs: Lifting security limitation of total HTTP headers > 8kB

[brianv0]

This is more of documenting an issue we had and a workaround we found.

We recently were debugging an issue where we were getting an HTTP 400, but not all the time.

We had some JWT tokens in headers and the total size was really close to 8192 bytes, but we would see things mostly work up to a point and then fail with an HTTP 400 and no other content (no Server header), we guess after setting some cookies which ended up setting the max header size over the limit. We weren't sure exactly where the 400 was coming from at first, but we narrowed it down to the proxy.

What we found was that Node has a new HTTP header size limit of 8192 bytes, from 80kB, starting late last year: https://nodejs.org/en/blog/release/v11.3.0/

In any case, we were able to workaround this by setting a new HTTP max header size with the NODE_OPTIONS var:

NODE_OPTIONS=--max-http-header-size=16384

We just wanted to document this experience in case anybody else ends up with a random HTTP 400 and they don't know why. For example, Azure AD has been known to issue large tokens with all the groups a user is a member of, and the total size of that can easily exceed 4kB. In some cases, that token may be included twice in HTTP headers by identity-aware proxies, for example, and easily surpass the 8kB limit.

Identity-aware/BeyondCorp proxies might be used in conjunction with the JWT authenticator for JupyterHub, for example.

[hernrup-modelon]

Thank you so much! That one was hard to track down.