Proxyman automatic setup does not capturing network requests from react server components in next.js

[tonyxiao]

This simple example unfortunately doesn't work :(

app/debug/page.tsx

export default async function MyPage() {
  const res = await fetch('https://httpbin.org/anything')
  return <div>My Page{res.text()}</div>
}

EDIT: Problem is request is being cached by next.js Solution is to use no-cache with fetch. See https://github.com/ProxymanApp/Proxyman/issues/2038#issuecomment-2143324535

[NghiaTranUIT]

@tonyxiao It's expected behavior because fetch from NodeJS doesn't respect the system HTTP Proxy.

To capture the fetch, you should use the Automatic Setup: https://docs.proxyman.io/automatic-setup/automatic-setup

It will open the pre-configured Terminal -> Start your NodeJS server here -> Proxyman will automatically capture all NodeJS request/response 👍

[tonyxiao]

I used the automatic set up and it still didn't work.

[tonyxiao]

That was in the title of the issue actually

[tonyxiao]

To be sure it works for if I just run a node script, the problem is when I try to run next.js it doesn't work

[NghiaTranUIT]

Ah, it's a special case: from react server components in next.js

My initial is a fetch from NodeJS server. Let me see

[NghiaTranUIT]

I guess it's because fetch from NextJS is a custom version, and it ignores all system Proxy.

You should manually add the Proxy to the fetch and trust the self-signed certificate. Here is the answer: https://stackoverflow.com/questions/72306101/make-a-request-in-native-fetch-with-proxy-in-nodejs-18

---

Proxyman can't capture it automatically because it's how NextJS works.

[tonyxiao]

I see. What is the underlying networking lib next js uses? And wouldn't proxyman hook into that?

Separately would you be able to create an example of how to use proxyman with next.js? Feels like it should be in the docs somewhere too.

[tonyxiao]

@NghiaTranUIT following up here. Would love to have an example that works with next.js and some better understanding why it doesn't work out of the box.

[NghiaTranUIT]

Sure, I will do it this weekend 👍

[NghiaTranUIT]

@tonyxiao turn out Proxyman can capture the fetch from Server Side Component as soon as you don't use the caching.

1. Start the Pre-configure Terminal on Tools -> Setup -> Automatic Setup
2. Use no-cache on fetch

export default async function MyPage() {
  const res = await fetch('https://httpbin.org/anything', { cache: 'no-cache' }) // no cache, or no request is sent to Proxyman
  return <div>My Page{res.text()}</div>
}

3. Start your NextJS server on this terminal
4. Done ✅

[tonyxiao]

Interesting , I will give that a try. Strange why is that tho, as my request should hit a cache miss as it runs the first time, which apparently is not the same as "no-cache"?

[tonyxiao]

Hmm just tested the first request does seem to be captured - it's only later requests that are cached that seems to be not captured.