unrolled / render

Go package for easily rendering JSON, XML, binary data, and HTML templates responses.
MIT License
1.94k stars 146 forks source link

BufferPool leads to wasted memory #85

Closed nsmith5 closed 3 years ago

nsmith5 commented 3 years ago

Problem

The current buffer pool appears to buffer writes for HTML templating. While the buffers are Reset upon resubmission to the pool, this doesn't free the underlying memory. The result in a long running web server is that the buffer pool contains ~64 * the largest template rendered.

This size can get unreasonable in applications like Gitea where some pages might contain sizable assets like pictures. My heap profiles show something like 500MiB of heap space just because of this.

Possible solutions

This buffer pool probably isn't needed at all. Simply push the concern of write buffering to the calling library. In fact, the most likely writer to be used with this library already buffers writes: http.ResponseWriter.

If a buffer pool is really needed, perhaps use something like this sized buffer pool that trims down buffers that grow past a certain size so that the pool can't grow unchecked.

unrolled commented 3 years ago

@nsmith5 I just merged your PR and rolled a release (v1.1.0). Do you want to try it out in production for a while before closing this issue?

nsmith5 commented 3 years ago

Sweet! Yeah, I'll try this out in my Gitea install and report on results

nsmith5 commented 3 years ago

Memory footprint drops as expected. This was merged into Gitea downstream here https://github.com/go-gitea/gitea/pull/15581