search

JoeDog / siege

Siege is an http load tester and benchmarking utility
GNU General Public License v3.0
5.97k stars 385 forks source link

Out of memory when sieging large data endpoint #221

Open dpetroff opened 1 year ago

dpetroff commented 1 year ago

I have an endpoint that responds with a lot of data. Think 500+ MB. When I try to siege it, siege runs out of memory and is killed:

siege -c 20 --no-parse http://localhost:8080/endpoint
** SIEGE 4.1.7-b2
** Preparing 20 concurrent users for battle.
The server is now under siege...Killed

I was hoping that the --no-parse option would meter the data in flight and shove it straight into the nether, but that doesn't seem to be the case.

JoeDog commented 1 year ago

Was this killed by the Out of memory killer? Because the kernel invokes that when your system is literally running out of memory. Since you're running siege on the same server as the web server, you need twice as much memory to accommodate a transaction.

--no-parser tells siege not to parse the page for elements to download like stylesheets and scripts. It still reads it completely before it tosses it.

On Fri, Apr 14, 2023 at 10:48 AM Dimo Petroff @.***> wrote:

I have an endpoint that responds with a lot of data. Think 500+ MB. When I try to siege it, siege runs out of memory and is killed:

siege -c 20 --no-parse http://localhost:8080/endpoint SIEGE 4.1.7-b2 Preparing 20 concurrent users for battle. The server is now under siege...Killed

I was hoping that the --no-parse option would meter the data in flight and shove it straight into the nether, but that doesn't seem to be the case.

— Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/221, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZS25E7OC24KWI4ZDYLXBFPTPANCNFSM6AAAAAAW6QU3J4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Jeff Fulmer 1-717-799-8226 https://www.joedog.org/ He codes

dpetroff commented 1 year ago

I'm not sure how to check if it was the OOM killer, but having that much memory on the machine for sieging purposes isn't really practical in this case. The server already has streaming implemented on the response side, so it does not actually require a lot of memory to serve the requests. That's actually the very property I wanted to test by sieging.