search

oven-sh / bun

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one
https://bun.sh
Other
73.5k stars 2.71k forks source link

Malformed_HTTP_Response error intermittently when using bun fetch proxy #12007

Closed zfcsoftware closed 3 weeks ago

zfcsoftware commented 3 months ago

What version of Bun is running?

1.1.15+b23ba1fe1

What platform is your computer?

Linux 6.5.0-35-generic x86_64 x86_64

What steps can reproduce the bug?

This error does not occur when no proxy is used. You need a proxy and a website that gives a 403 error for the error to occur. Just enter an ipv4 proxy in the code below and run it a few times.

[Node.js] Tested code:

import fetch from 'node-fetch';
import { HttpsProxyAgent } from 'https-proxy-agent';
fetch('https://nopecha.com/demo/cloudflare', {
    agent: new HttpsProxyAgent(`http://<proxy-username>:<proxy-password>@<proxy-ip>:<proxy-port>`)
})
    .then(res => res.text())
    .then(res => {
        console.log('Success Response');
    })
    .catch(console.error)

[Bun] Tested code:

import fetch from 'node-fetch';
fetch(`https://nopecha.com/demo/cloudflare`, {
    proxy: 'http://<proxy-username>:<proxy-password>@<proxy-ip>:<proxy-port>'
})
    .then(res => res.text())
    .then(res => {
        console.log('Success Response')
    })
    .catch(console.error)

node.js output: Always returns the page source

https://github.com/oven-sh/bun/assets/123484092/ff515bff-b032-46b6-95fb-6c68bbee5797

Bun output: Sometimes it returns the page source code, sometimes it returns the following error.

Malformed_HTTP_Response: Malformed_HTTP_Response fetching "https://nopecha.com/demo/cloudflare". For more information, pass `verbose: true` in the second argument to fetch()
 path: "https://nopecha.com/demo/cloudflare"

https://github.com/oven-sh/bun/assets/123484092/79c82179-2194-4149-b631-158ce1945cd1

What is the expected behavior?

When using bun fetch and proxy, requests returning a 403 response should return the page source code.

What do you see instead?

Sometimes it returns the page source code, sometimes it returns the following error. Malformed_HTTP_Response: Malformed_HTTP_Response fetching "https://nopecha.com/demo/cloudflare". For more information, pass verbose: true in the second argument to fetch() path: "https://nopecha.com/demo/cloudflare"

Additional information

In the other issue I opened, I explained the problems with axios proxy support, and in this issue I explained the problems with fetch. Right now I can't send a stable request using a proxy with bun, it always results in an error.

cardimajs commented 3 months ago

same here

eknkc commented 2 months ago

This has been a big issue for us too.

Since the intermittent issues made it impossible to use the fetch proxy param, I decided to use undici but bun can not access the ProxyAgent in undici.

Then I tried node's proxy agents. Not working.

Finally switched to node with undici's proxy agent.

marcfrankel commented 1 month ago

Same issue here as well. Would love to see this fixed

zfcsoftware commented 3 weeks ago

@Jarred-Sumner This problem persists.

https://github.com/user-attachments/assets/5d960750-47a1-4a64-909a-14d70707448a

cirospaciari commented 3 weeks ago

@Jarred-Sumner This problem persists.

2024-09-05.03-16-12.mp4

@zfcsoftware Can you try bun upgrade --canary? The fixes I added was on 1.1.27-canary.

zfcsoftware commented 3 weeks ago

bun upgrade --canary

this problem persists in the canary version with the panic bug.

https://github.com/user-attachments/assets/ccbd2c77-cc4d-4a36-b742-d6340531a263

zfcsoftware@zfcsoftware:~/Masaüstü/test$ bun run index.js 
Malformed_HTTP_Response: Malformed_HTTP_Response fetching "https://nopecha.com/demo/cloudflare". For more information, pass `verbose: true` in the second argument to fetch()
 path: "https://nopecha.com/demo/cloudflare"

zfcsoftware@zfcsoftware:~/Masaüstü/test$ 
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bun run index.js 
Malformed_HTTP_Response: Malformed_HTTP_Response fetching "https://nopecha.com/demo/cloudflare". For more information, pass `verbose: true` in the second argument to fetch()
 path: "https://nopecha.com/demo/cloudflare"

zfcsoftware@zfcsoftware:~/Masaüstü/test$ bun run index.js 
============================================================
Bun Canary v1.1.27-canary.78 (cd7f6a15) Linux x64
Linux Kernel v6.8.0 | glibc v2.35
CPU: sse42 popcnt avx avx2
Args: "bun" "run" "index.js"
Features: jsc fetch 
Builtins: "bun:main" "node:fs" "node:string_decoder" "node:util/types" "node-fetch" 
Elapsed: 899ms | User: 23ms | Sys: 9ms
RSS: 1.07GB | Peak: 52.40MB | Commit: 1.07GB | Faults: 0

panic: Segmentation fault at address 0x0
oh no: Bun has crashed. This indicates a bug in Bun, not your code.

To send a redacted crash report to Bun's team,
please file a GitHub issue using the link below:

 https://bun.report/1.1.27/lr2cd7f6a1AigCu/47kE+xyQ6u/7uD6jm53DsjmouDghyrqDksllBA2AA

Success Response
Yönerge kuraldışı (çekirdek döküldü)
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bun run index.js 
Malformed_HTTP_Response: Malformed_HTTP_Response fetching "https://nopecha.com/demo/cloudflare". For more information, pass `verbose: true` in the second argument to fetch()
 path: "https://nopecha.com/demo/cloudflare"
cirospaciari commented 3 weeks ago

https://bun.report/1.1.27/lr2cd7f6a1AigCu/47kE+xyQ6u/7uD6jm53DsjmouDghyrqDksllBA2AA

Thanks for testing will investigate

cirospaciari commented 3 weeks ago

@zfcsoftware can you try bunx bun-pr 13752? https://github.com/oven-sh/bun/pull/13752 should fix the crash, still investigating the Malformed_HTTP_Response but got a reproduction here, so working on that rn.

zfcsoftware commented 3 weeks ago

@zfcsoftware can you try bunx bun-pr 13752? #13752 should fix the crash, still investigating the Malformed_HTTP_Response but got a reproduction here, so working on that rn.

Thank you very much for your concern. After running the relevant command, I tested it again with bun run index.js and got the same panic error. So I cleaned out the /tmp and .cache folders, uninstalled bun-js completely and reinstalled it from scratch. When I tried to test it, I got the following output.

zfcsoftware@zfcsoftware:~/Masaüstü/test$ curl -fsSL https://bun.sh/install | bash
######################################################################## 100,0%
bun was installed successfully to ~/.bun/bin/bun 
Run 'bun --help' to get started
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bun upgrade --canary
[10.99s] Upgraded.

Welcome to Bun's latest canary build!

Report any bugs:

    https://github.com/oven-sh/bun/issues

Changelog:

    https://github.com/oven-sh/bun/compare/0a37423bafab6aebe803e5f1594cffa9796f2ae6...main
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bunx bun-pr 13752
67 |   exit_status: number;
68 |   artifact_paths: string;
69 |   agent: Agent;
70 |   created_at: string;
71 |   scheduled_at: string;
72 |   runnable_at: string;
                    ^
ShellError: Failed with exit code 1
 info: {
  "exitCode": 1,
  "stderr": "",
  "stdout": ""
}

      at new ShellError (12:16)
      at new ShellPromise (72:16)
      at BunShell (187:35)
      at module code (/tmp/bunx-1000-bun-pr@latest/node_modules/bun-pr/bun-pr.ts:72:58)

Bun v1.1.27-canary.78+cd7f6a158 (Linux x64)
cirospaciari commented 3 weeks ago

@zfcsoftware can you try bunx bun-pr 13752? #13752 should fix the crash, still investigating the Malformed_HTTP_Response but got a reproduction here, so working on that rn.

Thank you very much for your concern. After running the relevant command, I tested it again with bun run index.js and got the same panic error. So I cleaned out the /tmp and .cache folders, uninstalled bun-js completely and reinstalled it from scratch. When I tried to test it, I got the following output.

zfcsoftware@zfcsoftware:~/Masaüstü/test$ curl -fsSL https://bun.sh/install | bash
######################################################################## 100,0%
bun was installed successfully to ~/.bun/bin/bun 
Run 'bun --help' to get started
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bun upgrade --canary
[10.99s] Upgraded.

Welcome to Bun's latest canary build!

Report any bugs:

    https://github.com/oven-sh/bun/issues

Changelog:

    https://github.com/oven-sh/bun/compare/0a37423bafab6aebe803e5f1594cffa9796f2ae6...main
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bunx bun-pr 13752
67 |   exit_status: number;
68 |   artifact_paths: string;
69 |   agent: Agent;
70 |   created_at: string;
71 |   scheduled_at: string;
72 |   runnable_at: string;
                    ^
ShellError: Failed with exit code 1
 info: {
  "exitCode": 1,
  "stderr": "",
  "stdout": ""
}

      at new ShellError (12:16)
      at new ShellPromise (72:16)
      at BunShell (187:35)
      at module code (/tmp/bunx-1000-bun-pr@latest/node_modules/bun-pr/bun-pr.ts:72:58)

Bun v1.1.27-canary.78+cd7f6a158 (Linux x64)

Sorry that you got this error trying bunx bun-pr I was able to reproduce 3 bugs and all are fixed on the mentioned PR (including the Malformed_HTTP_Response) after the fix is merged this issue will be closed, I added tests to this scenarios, so this issues not appear anymore in the future when using proxies.

zfcsoftware commented 3 weeks ago

@zfcsoftware can you try bunx bun-pr 13752? #13752 should fix the crash, still investigating the Malformed_HTTP_Response but got a reproduction here, so working on that rn.

Thank you very much for your concern. After running the relevant command, I tested it again with bun run index.js and got the same panic error. So I cleaned out the /tmp and .cache folders, uninstalled bun-js completely and reinstalled it from scratch. When I tried to test it, I got the following output.

zfcsoftware@zfcsoftware:~/Masaüstü/test$ curl -fsSL https://bun.sh/install | bash
######################################################################## 100,0%
bun was installed successfully to ~/.bun/bin/bun 
Run 'bun --help' to get started
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bun upgrade --canary
[10.99s] Upgraded.

Welcome to Bun's latest canary build!

Report any bugs:

    https://github.com/oven-sh/bun/issues

Changelog:

    https://github.com/oven-sh/bun/compare/0a37423bafab6aebe803e5f1594cffa9796f2ae6...main
zfcsoftware@zfcsoftware:~/Masaüstü/test$ bunx bun-pr 13752
67 |   exit_status: number;
68 |   artifact_paths: string;
69 |   agent: Agent;
70 |   created_at: string;
71 |   scheduled_at: string;
72 |   runnable_at: string;
                    ^
ShellError: Failed with exit code 1
 info: {
  "exitCode": 1,
  "stderr": "",
  "stdout": ""
}

      at new ShellError (12:16)
      at new ShellPromise (72:16)
      at BunShell (187:35)
      at module code (/tmp/bunx-1000-bun-pr@latest/node_modules/bun-pr/bun-pr.ts:72:58)

Bun v1.1.27-canary.78+cd7f6a158 (Linux x64)

Sorry that you got this error trying bunx bun-pr I was able to reproduce 3 bugs and all are fixed on the mentioned PR (including the Malformed_HTTP_Response) after the fix is merged this issue will be closed, I added tests to this scenarios, so this issues not appear anymore in the future when using proxies.

It works great, thank you for taking care of this issue and fixing it. I think there is also a https://github.com/oven-sh/bun/issues/12005 issue with the bun-js proxy. I will use this local support for now.