Max concurrent of Bun.serve is limited to 50 rps

[SaltyAom]

What version of Bun is running?

1.0.12

What platform is your computer?

Darwin 23.1.0 arm64 arm

What steps can reproduce the bug?

I found that when await Promise with a long time to resolve (~1s) is significantly slower than it should be (from 100k to 300).

Using this code:

const sleep = (time: number) => new Promise(resolve => setTimeout(resolve, time))

Bun.serve({
    port: 3000,
    fetch: async () => {
        await sleep(1500)

        return new Response('Hi')
    }
})

Using this bombardier command:

bombardier --fasthttp -c 500 -d 10s http://localhost:3000/

Testing with both Bun.sleep and custom sleep function results in the same.

This behavior is likely not expected as HTTP should be working concurrently.

Using aha to inspect request per second, I found that request is limited to 50 request per second but sometimes jump every 1.5 seconds.

Changing sleep duration to 1000 causes the bottleneck, limited to exactly 50 requests per second.

This may mean that there’s some internal function in Bun that may be limiting the execution of concurrent requests.

Is this expected?

What is the expected behavior?

Should not be limiting to 50 max concurrent request per seconds

What do you see instead?

Being limited to 50 rps as describe above

Additional information

No response

[Jarred-Sumner]

This is an interesting bug

[Jarred-Sumner]

This regression began between Bun v1.0.7 and Bun v1.0.8.

[Jarred-Sumner]

~If we compare profiles between Bun v1.0.7 and Bun v1.0.8, we can see significantly more time spent in kqueue. ~

edit: this is incorrect, the code was different for each one because I had the wrong tab open

[Jarred-Sumner]

This issue seems to only apply to macOS.

macOS:

Summary:
  Success rate: 1.0000
  Total:        24.9805 secs
  Slowest:      0.0636 secs
  Fastest:      0.0251 secs
  Average:      0.0500 secs
  Requests/sec: 2001.5646

Linux:

Summary:
  Success rate: 1.0000
  Total:        10.0963 secs
  Slowest:      0.1008 secs
  Fastest:      0.0144 secs
  Average:      0.0500 secs
  Requests/sec: 19809.1068

Code:

var pending = [];
setInterval(() => {
  const next = pending;
  pending = [];
  for (let fn of next) {
    fn();
  }
}, 50);
const sleep = (time) => {
  const { promise, resolve } = Promise.withResolvers();
  pending.push(resolve);
  return promise;
};
var i = 0;
Bun.serve({
  port: 3000,
  fetch: async () => {
    const j = i++;
    console.time("Elapsed " + j);
    await sleep();
    console.timeEnd("Elapsed " + j);

    return new Response("Hi");
  },
});

[Jarred-Sumner]

Thinking about this more...this is actually expected behavior.

listen(2) receives a backlog. That backlog is determined mostly by your machine. Most macOS machines default to 128. That means it will enqueue up to 128 connections and any more than that, clients will report connection refused or timeout.

The backlog can be increased, but it doesn't necessarily mean more connections will go through.

There is also a limit on the number of open sockets available on the machine. On macOS, that usually is about 16k. On Linux, it's typically higher.

I do think there is some performance issue with timers in Bun that should be addressed in the future.

But I don't think this is actually unexpected behavior. I tried with Node and Deno and they behaved similarly. It's worth verifying other languages experience similar behavior, but based on what I know right now, it seems like this is expected behavior. Very happy to be proven wrong on this.