EDCD / EDDN

EDDN - Elite: Dangerous Data Network
https://eddn.edcd.io/
BSD 3-Clause "New" or "Revised" License
298 stars 59 forks source link

Gateway: Enforce a *plain-text* message size limit #171

Open Athanasius opened 2 years ago

Athanasius commented 2 years ago

We have now confirmed that compression does allow for larger plain-text messages to make it through bottle's body size check. It's only lookin at the 'raw', and thus compressed if applicable, size.

Normal gzip will compress things to 10-11% their original size, and thus the 1MiB bottle limit means potentially a ~10MiB plain-text message.

However it's possible to craft nasty gzip bombs that are small on the wire but decompress to much smaller. Thus we should see if zlib has any way to limit the amount of memory/buffer it will use in decompression and throw an exception if exceded.

Athanasius commented 2 years ago

Suggestion from A_D for how to enforce a plain-text limit:

import gzip, io
incomingstream: bytes = None # wherever this comes from
stream_file = io.bytesIO(incomingstream) # this can be skipped if incomingstream is already a file (eg a socket)
gz_file = gzip.GzipFile(fileobj=stream_file)

buffer = b''
while len(buffer) < MAX_BUFFER:
  buffer += gz_file.read(1024)

And Spansh has supplied a 1GB gzip bomb, see https://discord.com/channels/164411426939600896/205369618284544000/934051715957792818