Epic: Bypass PageCache for user data blocks

[jcsp]

From experiments, we believe that use of the userspace PageCache for user data blocks is a net-negative:

• Imposes significant synchronization overhead on getpage requests using the shared cache
• Displaces higher-value data (layer indices) from the cache
• Very low hit rate at scale when repeated accesses are usually absorbed by the postgres buffer cache.

Ideas / Initial Plan

• [ ] Mini-cache (~2 pages) on each request to enable common I/O patterns that read several sequential deltas
• [ ] Enable readers to optionally skip the page cache
• [ ] Pageserver config property to use the new I/O path
• [ ] Add/modify any metrics as needed to demonstrate the before/after behavior: we should see higher peak RPS and a higher cache hit rate for index pages.
• [ ] Remove caching of userspace pages.

Execution

### High-Level
- [x] bypass PageCache for data blocks of delta and image layers => done by Vlad's work for get_impl=vectored
- [ ] https://github.com/neondatabase/neon/pull/8105
- [ ] https://github.com/neondatabase/neon/issues/7418
- [ ] https://github.com/neondatabase/neon/issues/8184
- [ ] https://github.com/neondatabase/neon/issues/8183

[jcsp]

Next step:

• try removing materialized page cache & establish the impact -- what else needs to change?

[problame]

this week: plumb through RequestContext on read path

Also, Vlad informed me that the switch to vectored get for all Timeline::get means that we'll stop using the PageCache for user data blocks. It's starting to roll out into prod this week.

[problame]

This week:

• conclude VirtualFile RequestContext'ification
  • slab-allocated RequestContext
  • include Tenant/Shard/Timeline in RequestContext
  • use that for VirtualFile metrics
• review whether/where PS-PageCache-ing of data blocks is still relevant
• revise plan

[problame]

Update:

• we removed the materialized page cache https://github.com/neondatabase/neon/pull/8105
• the remaining data pages that are still in the PageCache is the InMemoryLayer pages
  • starting point for investigation: this panel here as well as the entire "Drill down into Content Kind" section in that dashboard
  • NB: above link compares the nightly pgbench run in staging the day before materialized page cache was removed and the day after. Prod data looks likely different.

[problame]

This time range here is instructive: in this time range, there is a high access rate coming from LayerFlushTask for InMemoryLayers which is displacing DeltaLayerBtreeNode layers which causes cache misses for WalReceiverConnectionHandler and to a lesser extent Compaction.

https://neonprod.grafana.net/d/fe7f056c-3ee1-49ef-a08d-e66055099396/pageserver-page-cache?orgId=1&from=1718937864602&to=1718957534079&var-datasource=HUNg6jvVk&var-hit_rate_drill_task_kind=WalReceiverConnectionHandler&var-hit_rate_drill_content_kind=InMemoryLayer&var-adhoc=neon_region%7C%3D%7Cus-east-2&var-adhoc=instance%7C%3D%7Cpageserver-4.us-east-2.aws.neon.build

[problame]

If it's just the LayerFlushTask perhaps this is the time to implement

• https://github.com/neondatabase/neon/issues/7418

[problame]

This week:

• merge & benchmark https://github.com/neondatabase/neon/issues/7418
• start working on https://github.com/neondatabase/neon/issues/8183

[problame]

Status update:

• gradually rolling into prod: https://github.com/neondatabase/neon/issues/7418
• scouted https://github.com/neondatabase/neon/issues/8183
  • seems like it's better done after https://github.com/neondatabase/neon/issues/8184
• https://github.com/neondatabase/neon/issues/8184
  • coding done
  • validation in staging, will build confidence over the next 1.5 weeks