How memory allocations will work in case of lots of connections?

[shivshankardayal]

Nice work on lwan. I had a question. Consider that lots of connections are made and there are handlers for each connection. Now when the handlers for these connections allocate memory on stack how will stack cope up with all this memory allocation?

[lpereira]

Thanks.

Handlers have to be careful to not use too much stack space, otherwise, it's in undefined behavior territory. (It's already sort of a grey area with coroutines, anyway.) You'll be lucky if the program crashes at that point, but it could corrupt other things, including the coroutine struct with the register state and whatnot.

I considered using GCC's split stack support (added originally for gccgo), but haven't gotten around to it yet.

Ideally, you'd allocate with coro_malloc() and forget about it (memory will be freed when the coroutine ends/request has been handled). For repeated small allocations (< 1024 bytes), allocation should be pretty quick.

On Wed, Jul 29, 2020, 07:08 Shiv Shankar Dayal notifications@github.com wrote:

Nice work on lwan. I had a question. Consider that lots of connections are made and there are handlers for each connection. Now when the handlers for these connections allocate memory on stack how will stack cope up with all this memory allocation?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lpereira/lwan/issues/285, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAADVGNCOBTKSWBZZ2STRF3R6AUMJANCNFSM4PLUKHSQ .

[shivshankardayal]

Thanks for the answer I was afraid of. The way I see is that even though you need only 500KB memory for 10k concurrent connections in real-world the coroutines will need their own stack. Now the bigger problem is if we allocate the 8 MB default stack(that is stack size for a process on my 64-bit Linux) for one coroutine we are looking at 80G RAM for 10k connections(even if we allocate 512 KB we are looking at 5G of RAM but I do not see this problem going away with large number of connections no matter which design we choose). It becomes even worse when you try to implement something like websocket where the connection has to be kept open for long periods of time.

So what I think is that Seastar may have some solution for this kind of problem though I have not investigated that.

[lpereira]

The stack of each coroutine is just a few kilobytes, it's not 8MB. The size isn't the same as the size of a pthread stack, but each coroutine has its own stack. (That 500kb/10k figure should be updated as it's using a little bit more these days, though.)

But, yes, that's the main issue with stackful coroutines. I experimented with stackless ones, where I could reuse the same stack for all of them but keep only their state in a struct somewhere, but it's not really trivial. It's cumbersome in C, to say the least. With h C++20 and it's coroutines, it's possible to do this without breaking a sweat though.

On Wed, Jul 29, 2020, 16:16 Shiv Shankar Dayal notifications@github.com wrote:

Thanks for the answer I was afraid of. The way I see is that even though you need only 500KB memory for 10k concurrent connections in real-world the coroutines will need their own stack. Now the bigger problem is if we allocate the 8 MB default stack(that is stack size for a process on my 64-bit Linux) for one coroutine we are looking at 80G RAM for 10k connections(even if we allocate 512 KB we are looking at 5G of RAM but I do not see this problem going away with large number of connections no matter which design we choose). It becomes even worse when you try to implement something like websocket where the connection has to be kept open for long periods of time.

So what I think is that Seastar may have some solution for this kind of problem though I have not investigated that.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lpereira/lwan/issues/285#issuecomment-665978152, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAADVGIDNDZNC5A66JRQ3DLR6CUVDANCNFSM4PLUKHSQ .

[lpereira]

Closing this issue; if you have more questions, feel free to open another or reopen this one.