fix(pageserver): make delta split layer writer finish atomic

[skyzh]

Problem

similar to https://github.com/neondatabase/neon/pull/8841, we make the delta layer writer atomic when finishing the layers.

Summary of changes

• put_value not taking discard fn anymore
• finish decides what layers to keep

Checklist before requesting a review

• [ ] I have performed a self-review of my code.
• [ ] If it is a core feature, I have added thorough tests.
• [ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• [ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• [ ] Do not forget to reformat commit message to not include the above checklist

[github-actions[bot]]

5281 tests run: 5067 passed, 0 failed, 214 skipped (full report)

---

Flaky tests (2)

#### Postgres 17 - `test_readonly_node_gc`: [debug-x86-64](https://neon-github-public-dev.s3.amazonaws.com/reports/pr-9048/11444733437/index.html#suites/3ccffb1d100105b98aed3dc19b717917/8fc21313bad410e2/retries) - `test_timeline_offloading[False]`: [release-arm64](https://neon-github-public-dev.s3.amazonaws.com/reports/pr-9048/11444733437/index.html#suites/d1acc79950edeb0563fc86236c620898/73028fa6f54a6345/retries)

Code coverage* (full report)

• functions: 31.4% (7653 of 24388 functions)
• lines: 48.9% (60198 of 123134 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
b4ae4c4857f9ef3e144e982a35ee23bc84c71983 at 2024-10-22T22:17:10.210Z :recycle:}