denosaurs / bench

📊 Comparing deno, node and bun HTTP frameworks
MIT License
372 stars 33 forks source link

Bun is 2x faster than Deno #86

Closed aquapi closed 7 months ago

aquapi commented 1 year ago

I have benchmarked Deno a few times on my machine and many benchmarks Deno is 2x slower than Bun Maybe you can try use another benchmarking tool like wrk or bombardier to see the result.

Funding

Fund with Polar

aquapi commented 1 year ago

You can see these benchmarks

JasperVanEsveld commented 1 year ago

There are quite a lot of benchmarks around and not a lot seem to agree with eachother, let alone that bun is 2x faster than deno. For your bunsvr/benchmark, where can I see that bun is 2x faster? I can only see different frameworks not runtimes in the results.

Looking at SaltyAom/bun-http-framework-benchmark there is an open issue that shows windows could impact the results. The Linux results in that thread also wildly vary, one where node + uws wins (uws is also what bun uses). Also the results from this user shows that deno with oak is within ~11%* of bun (edit: missed that JSON results were bugged, thanks @aquapi). Even though in the results on this repo bun is 2.7x faster than deno with oak. Like I said, the benchmarks don't seem to agree on much...

I am not involved with this repository whatsoever, but I think it would be appreciated to find a problem with their methodology instead of dropping a claim like this. For example:

Which I think would all be valid questions and would be easier to follow up on. Personally I would love to know version numbers and which OS is used for example. This would also make it easier to validate these results.

aquapi commented 1 year ago

For simple benchmarking, wrk or wrk2 should be fine. It provides a Lua API for you to work with and wrk is the most performant benchmarking tools.

The result mentioned above has an issue in POST /json test, which made the result inaccurate. @JasperVanEsveld

icetbr commented 1 year ago
bun:  89,218.22 req/s
deno: 70,772.89 req/s
node: 28,542.56 req/s

Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz Linux Mint 21.2 kernel 5.15.0-82-generic

node v20.3.1 bun 1.0.0 deno 1.36.1 (release, x86_64-unknown-linux-gnu) v8 11.6.189.12 typescript 5.1.6

deno.ts

Deno.serve({ port: 3000 }, () => new Response("Hello, Bench!"));

bun.ts

export default { port: 3000, fetch: () => new Response('Hello, Bench!') };

node.js

require('http').createServer((request, response) => (response.write('Hello, Bench!'), response.end())).listen(3000);

bombardier --fasthttp -c 500 -d 10s http://localhost:3000/

All the best deno frameworks stay around 67,689.84 req/s (oak)

aquapi commented 1 year ago

WSL may have some impact on the result as it has some limitation as a virtual machine. Idk if Deno has improved or not, but still.

I suggest you wrap the node handler with setImmediate and see the result

icetbr commented 1 year ago

Like this? No difference

const handler = response => () => {
    response.write('Hello, Bench!');
    response.end();
}
require('http').createServer((request, response) => setImmediate(handler(response))).listen(3000);
aquapi commented 1 year ago

@icetbr test on MacOS or other distro than Linux Mint Bun seems to be performing bad in Linux Mint for some reasons? (I suggest using Ubuntu)

icetbr commented 1 year ago

Mint is Ubuntu. I don't have MacOS, sorry. My benchmarks seems consistent with some articles I found online by others. Post your results if you can.

aquapi commented 1 year ago

@icetbr bunsvr/benchmark is mine

icetbr commented 1 year ago

Cool! I can only see bun frameworks for now, are you updating the repo?

aquapi commented 1 year ago

@icetbr Can you give me a framework code because I don't want to install Deno lint

icetbr commented 1 year ago

Which framework? Deno itself? I've used the ones from here https://github.com/SaltyAom/bun-http-framework-benchmark/tree/main/src/deno

And pure deno

Deno.serve({ port: 3000 }, () => new Response("Hello, Bench!"));

aquapi commented 1 year ago

@eliassjogreen Why are benchmark results flunctuating on every single update? You should check the benchmarking tool again or use wrk2.

aquapi commented 11 months ago

@icetbr Bun performs better in Ubuntu than in Linux Mint

icetbr commented 11 months ago

Do you have the stats? That would be unusual, as I'm not aware of any performance difference between any program between any linux distro.

aquapi commented 11 months ago

@icetbr if you have Mint you can for example test node:test with bun:test. On my ubuntu machine bun is 3x faster It may varies between distros as I see other results on Mint states that node:test is 3x faster

icetbr commented 11 months ago

Do you have any reference of this? It makes no sense, its like saying strawberry ice cream is colder then chocolate ice cream. Mint IS Ubuntu. Even Arch, which is way different, there is no performance difference. I don't doubt your results, but I'm curious of the circumstances of the test.

Even MacOs has the same performance. At least it used to be, I stopped following the benchmarks of dev environments a while ago, when i was deciding between a PC vs Mac for Node development.

aquapi commented 11 months ago

@icetbr See strager YT video

Shyam-Chen commented 7 months ago

https://github.com/Vanilla-IceCream/fastify-uws?tab=readme-ov-file#benchmarks

Version Language Router Requests/sec
bun 1.0.33 TypeScript/Bun ✗ 157,740.9324
deno 1.41.3 TypeScript/Deno ✗ 124,009.3235
node 20.11.1 JavaScript/Node ✗ 78,002.1550

Bun is 2x faster than Node, not Deno.

aquapi commented 7 months ago

@Shyam-Chen It was