pfalcon
commented
6 years ago I'd like to start with the approach 2 as proof of concept, promising minimal data space overhead, and potentially benefiting Zephyr beyond just sockets subsystem. So, we'd need to k_poll() on pkt pool, which is backed struct k_mem_slab. And here's caveat: this structure is not k_poll'able. That's why I'm saying that going this way would benefit Zephyr, but minimizing adhoc split between k_poll'able and just thread-blocking structures.
AndreyDodonov-EH
rlubos
carlescufi
Currently, zsock_poll(POLLOUT) implemented as unconditionally returning this flag. This means that if there're no free packet/data buffers, then user application will be effectively in busy-polling loop: POLLOUT will be returned, and app will try to send() to a socket and will get EAGAIN, will call poll() and immediately get POLLOUT, repeated.
There can be different solutions to this, of different level of overhead, "precision" (i.e. how big there's a chance still that POLLOUT will be returned from poll(), yet EAGAIN from send()), and changes required to Zephyr kernel and/or IP stack code. Btw, the fact that overhead and/or changes to code beyond BSD Sockets layer would be required, was the reason why such support wasn't implemented initially and postponed until later.
Two solutions on roughly opposite sides of "precision" spectrum:
The first idea is to limit number of outgoing inflight packets for a socket. If socket is currently sending out pkt, it's not POLLOUT'able. Implementation of that would require adding a semaphore to each socket (aka net_context, because socket is nothing but a net_context). send() would take semaphore, "sent" callback would give it. That alone doesn't guarantee high precision though, as a socket might not have an inflight pkt, and yet pkt pool might not have a free pkt either. To solve precision "problem", would need to associate a static outgoing pkt buffer with each socket. That's another field in socket structure, and (much) higher pressure on pkt pool. This still isn't going to resolve too much though, due to 2-level net allocation structure of Zephyr: to send out a pkt, it's required to allocate net_pkt, then allocate one of more net_buf, and attach to net_pkt. To guarantee something, would require to keep static net_pkt and net_buf per each socket.
Another idea is to poll pkt pool for free pkt slots. By definition, this method is not precise, because several sockets may compete for a single pkt buffer. And it's even less precise due to 2-level alloc structure, described above.