search

neondatabase / neon

Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.
https://neon.tech
Apache License 2.0
14.78k stars 430 forks source link

Use streaming in debug_dump responses #5710

Closed petuhovskiy closed 11 months ago

petuhovskiy commented 11 months ago

We have a /debug_dump HTTP endpoint in safekeepers, to collect debug info about all timelines. It produces a large JSON object, which is currently held in memory.

Current implementation is here: https://github.com/neondatabase/neon/blob/main/safekeeper/src/debug_dump.rs#L143-L201

It is possible to rewrite this logic to use streaming HTTP responses (we've already done a similar thing for /metrics here) and we also should use a streaming JSON encoding (should be possible to do by implementing Serialize trait for iterator).

Marking this as good_first_issue since it is can be done by anyone who can write Rust.

thread in slack: https://neondb.slack.com/archives/C04KGFVUWUQ/p1690470241725729

duguorong009 commented 11 months ago

I would like to pick this issue! @petuhovskiy