search

unjs / nitro

Next Generation Server Toolkit. Create web servers with everything you need and deploy them wherever you prefer.
https://nitro.unjs.io
MIT License
5.89k stars 496 forks source link

Cached routes significantly decrease throughput #2555

Closed InstantlyMoist closed 3 months ago

InstantlyMoist commented 3 months ago

Environment

Node v22.2.0 Nitro latest (as of posting)

Reproduction

https://github.com/InstantlyMoist/nitro-throughput

Describe the bug

Currently it appears that using a cached event handler reduces the throughput of a route by around 40% this is tested with

https://github.com/fomalhaut88/winrk but can also be tested with wrk for linux.

My findings (based on reproduction) For all endpoints I used the same settings (t = 4, d = 10, c = 200), tested 3 times

/simple = ~16k requests per second /simple_cached = ~11k requests per second /big_payload = ~100 requests per second /big_payload_cached = ~60 request per second

While nitro's throughput seems to significantly decrease with a higher payload, it seems to decrease even further with the use of a cached event route.

Additional context

No response

Logs

No response

manniL commented 3 months ago

(Linking Discord discussion here)

pi0 commented 3 months ago

Hi dear @InstantlyMoist. You should use caching when the computation cost is more than cache retrieval like when making a costly database query or long time fetch.

In the /simple example, computing a simple new Date() is actually cheaper than more instructions needed to store and retrieve the cache.

In the /big_payload case, we are actually not computing anything. Returning a JS object is also significantly cheaper than serializing it to store in the cache and deserializing before sending response.