oven-sh / bun

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one
https://bun.sh
Other
74.12k stars 2.76k forks source link

Empty Response with 0 KB Body When Using 'Authorization' Header in Bun Fetch #4977

Closed prstance closed 3 weeks ago

prstance commented 1 year ago

What version of Bun is running?

1.0.0+ea56182c5a45be7978de8521f6cdaee0e0772434

What platform is your computer?

Linux 5.15.90.1-microsoft-standard-WSL2 x86_64 x86_64

What steps can reproduce the bug?

Steps to Reproduce the Bug:

  1. Create a JavaScript script to perform an HTTP fetch request with an 'Authorization' header.
  2. Use a specific URL that requires authentication, such as an API endpoint.
  3. Include the 'Authorization' header in the request with valid credentials. fetch("xxx", { "headers": { "Authorization": "xxx", "User-Agent": "xxx" } }).then(response => response.text()).then(json => console.log(json))
  4. Check the response received from the fetch on Bun, and see the difference with NodeJs. image

Bug Explanation: When performing the fetch request with the 'Authorization' header included, Bun returns a response with an empty body, which has a size of 0 KB. This behavior is unexpected and not in line with the normal behavior of other HTTP client libraries and tools like Node.js fetch, Python, PHP, and curl.

The bug appears to be specific to Bun when handling requests with the 'Authorization' header. It results in an incomplete response, making it impossible to retrieve the expected data from the server when authentication is required.

What is the expected behavior?

The expected behavior is to receive a response from the server with the correct body data. The response body should contain the data or content that corresponds to the requested URL and authentication credentials, allowing the script to process and utilize the received information as needed. image

What do you see instead?

Instead of receiving the expected response with the relevant data, when using Bun to make a fetch request with the 'Authorization' header included, the observed behavior is as follows:

  1. The response received from the server contains an empty body.
  2. The size of the response body is 0 KB, indicating that no data or content has been successfully retrieved.
  3. This behavior is inconsistent with other HTTP client libraries and tools, such as Node.js fetch, Python, PHP, and curl, which return the correct response data when the 'Authorization' header is provided.

In summary, the issue with Bun is that it returns an empty response body (0 KB) when the 'Authorization' header is used, which is not the expected behavior for a functioning HTTP client.

Additional information

  1. The issue with Bun has been tested with multiple URLs that require authentication using the 'Authorization' header. The problem persists consistently across various endpoints, indicating that it is not specific to a particular API or server.
  2. Other HTTP client libraries and tools, including Node.js fetch, Python, PHP, and curl, do not exhibit this issue and return the expected response data when the 'Authorization' header is included.
  3. It would be helpful to investigate the internal handling of headers and response data in Bun when an 'Authorization' header is present to identify the root cause of this behavior.
  4. The issue has been reported on the dedicated GitHub page for Bun, providing further details and context for developers to investigate and address the problem effectively.
Vexcited commented 1 year ago

This is a solution I found for/with him, after chatting on Discord, had a similar issue with WebSockets that I solved the same way

So basically, we simply make our own request from scratch using the node:tls (or node:net) module. By doing this way, it fully works and gives the result expected.

import tls from "node:tls";

const create_headers = (path: string, headers: Record<string, string>, body = ""): string => {
  const head = [
    `GET ${path} HTTP/1.1`,
    // Add every items from the headers object.
    ...Object.entries(headers).map(([key, value]) => `${key}: ${value}`),

    "", // There's a linebreak between the headers and the request's body. 

    body
  ];

  return head.join("\r\n");
};

const request = (url: URL, options = {
  method: "GET",
  headers: {},
  body: ""
}) => new Promise<string>((resolve, reject) => {
  const port = url.protocol === "https:" ? 443 : 80;
  const socket = tls.connect(port, url.hostname);

  socket.on("data", (message: Buffer) => {
    const response = message.toString("utf8");
    resolve(response);
    socket.end();
  })

  socket.on("connect", () => {
    socket.write(create_headers(url.pathname + url.search, options.headers, options.body))
  });
});

const url = new URL("https://..../profiles");

const response = await request(url, {
  method: "GET",  
  body: "",
  headers: {
    Host: url.host,
    Authorization: "Bearer XXXXXXXXX"
  },
});

const body = response.split("\r\n\r\n")[1];
const json = JSON.parse(body);

console.log(json);

We don't know the reason for the fetch issue, yet, but it would be cool if anyone could investigate on this (wonder if it's related or not with my issue https://github.com/oven-sh/bun/issues/4529)

jzila commented 2 months ago

I'm seeing this same issue, in Bun 1.1.26. It's not consistent, happening seemingly at random, with higher probability after several calls in a row. I'm calling the userinfo API on Google's oauth endpoint:

30T15:09:26.692Z", message: "GET request to https://openidconnect.googleapis.com/v1/userinfo, with query params {} and headers {"Authorization":"Bearer <redacted>"}", file: "/home/bun/app/dist/.next/server/chunks/226.js:10:154780" sdkVer: "20.0.4"}
2024-08-30T15:09:26.756Z com.supertokens {t: "2024-08-30T15:09:26.756Z", message: "Received response with status 200 and body <empty>", file: "/home/bun/app/dist/.next/server/chunks/226.js:10:154780" sdkVer: "20.0.4"}

If I curl this same endpoint with the same credentials, it always works.

Jarred-Sumner commented 2 months ago

@jzila can you try setting BUN_CONFIG_VERBOSE_FETCH=1 and see if the logs miss the headers too? This will only work if it’s using fetch and not the grpc http 2 client

if it was using http2, that would likely be related. We do have tests for the http2 module but it’s different enough from the rest of the code to be more likely to have bugs like this

jzila commented 2 months ago

It's HTTP 1.1:

> HTTP/1.1 GET https://openidconnect.googleapis.com/v1/userinfo
> accept: application/json
> authorization: Bearer <redacted>
> Connection: keep-alive
> User-Agent: Bun/1.1.26
> Host: openidconnect.googleapis.com
> Accept-Encoding: gzip, deflate, br

< 200 OK
< Date: Tue, 03 Sep 2024 13:42:29 GMT
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: Mon, 01 Jan 1990 00:00:00 GMT
< Content-Type: application/json; charset=utf-8
< Vary: Origin
< Vary: X-Origin
< Vary: Referer
< Content-Encoding: gzip
< Server: ESF
< X-XSS-Protection: 0
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
< Transfer-Encoding: chunked
jzila commented 2 months ago

This appears to be fixed in 1.1.27