search

sammhicks / picoserve

An async no_std HTTP server suitable for bare-metal environments, heavily inspired by axum
MIT License
178 stars 20 forks source link

Support chunked transfer encoding #44

Open Dominaezzz opened 3 weeks ago

Dominaezzz commented 3 weeks ago

This allows either client or server send requests/responses without having to specify a Content-Length upfront. This works by specifying a Transfer-Encoding of chunked, then the sender can send chunks of data and terminate the transfer by sending an empty chunk. I'm particularly interested in chunked responses.

My specific use case for this is streaming JPEGs from an ESP32-S3. When the ESP32-S3 is pulling JPEGs from a camera, it (usually) does not know the size of the image until the entire image is received into memory. Receiving the entire image into memory creates two problems; The first is latency, the image data should ideally be sent out ASAP after it's gotten from the camera. The second is image size, if the quality and width/height are large enough, the chip will be unable to fit the image in memory, which means it simply can't stream the image. Ideally the chip should be able to send out the image data ASAP to reduce the memory requirements.

I'm not sure what this should look like yet. Perhaps some kind of ChunkWriter I guess but I'm not sure how the terminating chunk should be handled yet.

(This library is fantastic btw! Thanks for building iand sharing it!)

sammhicks commented 3 weeks ago

I've written a draft implementation (see 8033a31), what do you think?

Dominaezzz commented 3 weeks ago

It looks great! It's more/less what I imagined. Thanks for throwing an implementation together so quickly.

My only comment is that it'd be convenient if the ChunkWriter itself also implemented the Write trait, and that way I could use the write! macro with it.

sammhicks commented 2 weeks ago

I've implemented write_fmt for ChunkWriter, which means that you'll be able to use the write macro, even though technically ChunkWriter doesn't implement Write. Does that meet your request?

https://github.com/sammhicks/picoserve/commit/f66c158b7a8b3fdb3f8b360e573d6074e9f782f2#diff-6186c6610b78b8b16d8526d10c5a2cfe8882b95b873e7a36a989de3e7b581599R34

Dominaezzz commented 2 weeks ago

Yes that's perfect thanks!

Dominaezzz commented 2 weeks ago

I've just used this in my application and it works like a charm. Last thing I'll ask for is a flush() method to ensure all previous chunks have been sent.

sammhicks commented 2 weeks ago

When you call ChunkWriter::finalize, the writer is flushed, other than that you shouldn't need to worry about flushing the buffers. What scenario would you like to flush the buffers partway through writing a response?

Dominaezzz commented 2 weeks ago

In my case I'm doing both chunked encoding and mixed-replace. So I have an infinite chunked steam, and I want to flush after every replacement (a complete jpeg).