snapframework / snap-server

A fast HTTP server library, which runs Snap web handlers.
http://snapframework.com/
BSD 3-Clause "New" or "Revised" License
196 stars 86 forks source link

Remove artificial maximum chunk-size limitation #130

Closed hvr closed 4 years ago

hvr commented 4 years ago

snap-server had this limitation in place since at least 2012; it's existence has been justified as an anti-DoS measure. However, since the HTTP specification does not state any maximum chunk-size, there's clients and proxies that rightfully do not make any effort to keep the chunk-size smaller than snap-server's arbitrary 256KiB max-chunk-size limit nor is does there appear to be any consensus nowadays that oversized chunk-size attacks are a relevant vector; so many modern HTTP implementations seem to handle oversized chunks just fine (after all, if we support large bodies via content-length, then why shouldn't we also be able to support a chunked-transfer where all payload is placed into a single chunk?)

This patch swaps out the old implementation with a stripped down version of the battle-tested one from the http-streams package where the module that contains the chunked-transfer parser states

-- Copyright © 2012-2018 Operational Dynamics Consulting, Pty Ltd
--
-- The code in this file, and the program it is a part of, is
-- made available to you by its authors as open source software:
-- you can redistribute it and/or modify it under the terms of
-- the BSD licence.
--
-- Significant portions of this file were written while studying
-- the HTTP request parser implementation in the Snap Framework;
-- snap-core's src/Snap/Internal/Parsing.hs and snap-server's
-- src/Snap/Internal/Http/Parser.hs, and various utility functions
-- have been cloned from there.

So we've come full circle...

hvr commented 4 years ago

fwiw, the new chunked transfer parser also appears to be more heap-allocation efficient than the original one by almost a factor 2x -- but this needs more accurate benchmarking to confirm.