Preflight cache middleware opt-out not working for rewrite of dynamic routes

[ambrauer]

Link to the code that reproduces this issue

https://github.com/ambrauer/next-prefetch-cache-issue

To Reproduce

1. Run yarn install, yarn build, then yarn start
2. Open the home page and click on the "Dynamic" nav link (/1)

Current vs. Expected behavior

Current behavior "Page 1" content is loaded on navigation.

Expected behavior "Page 2" content is loaded on navigation.

If you reload the page, you'll see "Page 2" loaded properly.

Additional details PR https://github.com/vercel/next.js/pull/32767 introduced a way for middleware to opt-out of next/link prefetch caching by setting the x-middleware-cache header to no-cache. However, this doesn't seem to be working when using NextResponse.rewrite with dynamic routes (in this example, [id].tsx with ids 1 and 2). The middleware example simulates a change in behavior between prefetch and navigation by rewriting to different paths based on the purpose header === prefetch.

Compare this behavior with static routes (in this example, a.tsx and b.tsx) by clicking on the "Static" nav link (/a). You'll see the "Page B" content is loaded on navigation as expected.

Also compare this behavior with use of redirect instead of rewrite. If you change this line in the middleware from

const response = NextResponse.rewrite(new URL(rewriteUrl, request.url));

to

const response = NextResponse.redirect(new URL(rewriteUrl, request.url));

and try the same "Dynamic" nav link, you'll see "Page 2" content is loaded on navigation as expected.

Provide environment information

Operating System:
  Platform: win32
  Arch: x64
  Version: Windows 11 Pro
  Available memory (MB): 32432
  Available CPU cores: 16
Binaries:
  Node: 20.11.1
  npm: N/A
  Yarn: N/A
  pnpm: N/A
Relevant Packages:
  next: 15.0.0-canary.29 // Latest available version is detected (15.0.0-canary.29).
  eslint-config-next: N/A
  react: 19.0.0-beta-04b058868c-20240508
  react-dom: 19.0.0-beta-04b058868c-20240508
  typescript: 5.1.3
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Middleware, Navigation, Pages Router

Which stage(s) are affected? (Select all that apply)

next start (local)

Additional context

Note this is Pages router; I have not verified if the issue exists for App router. See also related discussion here: https://github.com/vercel/next.js/discussions/43675