search

cloudflare / foundations

Cloudflare's Rust service foundations library.
https://blog.cloudflare.com/introducing-foundations-our-open-source-rust-service-foundation-library
BSD 3-Clause "New" or "Revised" License
1.25k stars 51 forks source link

Buffer JSON slog messages #62

Closed hcstern closed 1 month ago

hcstern commented 1 month ago

serde_json will happily do a write syscall for every token it serializes. Add a small 100-character BufWriter buffer to prevent excessive syscalls.

Note that serde_json does not appear to call Write::flush so short lines will not be printed until the buffer is full.

hcstern commented 1 month ago

The non-flushing behavior is probably unexpected and might make debugging confusing but I don't think is a problem in prod. Would it make sense to only turn on the bufwriter behavior with a release profile?

inikulin commented 1 month ago

The non-flushing behavior is probably unexpected and might make debugging confusing but I don't think is a problem in prod. Would it make sense to only turn on the bufwriter behavior with a release profile?

I think it's fine if we enable https://docs.rs/slog-json/latest/slog_json/struct.JsonBuilder.html#method.set_flush

hcstern commented 1 month ago

The non-flushing behavior is probably unexpected and might make debugging confusing but I don't think is a problem in prod. Would it make sense to only turn on the bufwriter behavior with a release profile?

I think it's fine if we enable https://docs.rs/slog-json/latest/slog_json/struct.JsonBuilder.html#method.set_flush

Nice catch, thanks! I'll increase the buffer size as well.