search

tmattio / dream-encoding

Encoding primitives for Dream applications.
ISC License
16 stars 2 forks source link

Appears not to support streaming #1

Open aantron opened 3 years ago

aantron commented 3 years ago

e.g.:

https://github.com/tmattio/dream-encoding/blob/dde420426ba98da2bc7d26b0d67c3f0f2eb7b7ab/lib/dream_encoding.ml#L194-L195

This appears to read the entire body into memory, and then compress it. I assume (perhaps wrongly) that the underlying compression algorithms support streaming, and it seems like streaming would be more beneficial for precisely the kind of responses that you wouldn't want to load into memory an extra time (large ones).

tmattio commented 3 years ago

assume (perhaps wrongly) that the underlying compression algorithms support streaming

It does! In fact, the compression function (which is almost identical to decompress's documentation example) creates a stream from the given string, so refactoring this to work on stream responses should be trivial 🤞

I'll get to it as soon as I have some spare time

aantron commented 3 years ago

I'll get to it as soon as I have some spare time

No rush! And I think you will likely need some improvements in Dream's streaming APIs to get it done, so do bug me :)