smoltcp as Redox network stack

[batonius]

Context: I've been working on reimplementing Redox's network stack with smoltcp, and it looks good so far. In the process, I've come up with a number of issues/ideas/proposals I want to discuss.

New features:

1. smoltcp doesn't support loopback interface, an attempt to connect to 127.0.0.1 results in a flood of ARP requests asking for the MAC address of 127.0.0.1. The loopback example solves this by using a loopback device, which is not an option for smolnetd. I think smoltcp should support several devices, each with several subnets assigned to them. This way we would be able to add loopback interface by adding a loopback device with 127.0.0.1/8 assigned to it.
2. smoltcp looks like a good place to implement DHCP since it already handles ARP, but I'm not sure how it should look like. Currently, Redox uses dhcpd daemon for this.
3. ICMP sockets aka IPPROTO_ICMP would be quite handy for implementing a version of ping able to work without suid, but I'm not sure if they're useful for anything besides ping. Right now Redox has icmpd daemon.
4. smoltcp doesn't check uniqueness of local ports when binding sockets and doesn't generate them when bind is called with port == 0, so smolnetd implements its own ports set. I think some form of it should be integrated into smoltcp itself, probably as part of the packet dispatch effort.

Improvements:

1. It would be nice to be able to manipulate TTL of a socket since it's supported by TcpStream and UdpSocket.
2. The current TCP accept/listen mechanism, while working, seems too limited. Namely, since smolnetd has to reopen a socket every time it accepts a connection, it's impossible to connect to the host during this period. I have no concrete proposal for this yet.
3. I think ARP requests rate should be limited, it's too easy to get smoltcp flooding the network with ARP packets.

Optimizations:

1. I still want to see #19 implemented.
2. Currently smolnetd iterates over all the sockets after each poll to see if it should send an event/resume a blocked call. It would be handy if EthernetInterface::poll produced some kind of list of the sockets which state changed during the call.
3. Right now smolnetd has to call SocketSet::prune after each SocketSet::release to free sockets, which means iteration over all the sockets in the set. I understand we can't free sockets in release because TCP sockets require shutdown, but maybe we should initiate the shutdown in release and then have SocketSet threat closed TCP sockets with refs == 0 as empty?

@whitequark Of all the issues, I think loopback interface is the most pressing right now, so I'd like to know what you think would be the best way to approach it.

cc @little-dude , @jackpot51

[whitequark]

I think smoltcp should support several devices, each with several subnets assigned to them. This way we would be able to add loopback interface by adding a loopback device with 127.0.0.1/8 assigned to it.

Correct.

smoltcp looks like a good place to implement DHCP since it already handles ARP, but I'm not sure how it should look like. Currently, Redox uses dhcpd daemon for this.

What I want to do is to extract the type definitions in wire (so IpAddr and its friends) into a separate crate, let's say smolnet. (It's not really an option to be 100% compatible with libstd because it provides e.g. blocking address resolution interfaces, and I think my version is anyhow more elegant.) That crate would have an std feature that adds conversion to/from libstd types, but otherwise stand on its own.

Then, I would add yet another crate, let's say smolsockets. This would provide a socket abstraction that would wrap around either (a) libstd sockets or (b) smoltcp sockets.

Then, protocols could be implemented on top of smolsockets, tested either directly on the host or over smoltcp (which gives us good options for QA), and most imporotantly, it's possible to e.g. use DHCP either (a) directly on top of smoltcp sockets or (b) on top of the libstd abstractions provided by e.g. redox.

ICMP sockets aka IPPROTO_ICMP would be quite handy for implementing a version of ping able to work without suid, but I'm not sure if they're useful for anything besides ping. Right now Redox has icmpd daemon.

Traceroute. Just add them, it's not a huge cost, and we have socket features now anyway. The reduction in potential data leakage is sweet.

I think some form of it should be integrated into smoltcp itself, probably as part of the packet dispatch effort.

I agree. I actually think that smoltcp should always use maps for dispatch. Except, on bare metal without allocation it should be a linearized O(n) map like the one currently used in SliceArpCache, and with allocation it should just be a BTreeMap. So there'll be a ManagedMap. This means that port uniqueness / ephemeral generation will become trivial.

It would be nice to be able to manipulate TTL of a socket since it's supported by TcpStream and UdpSocket.

Sure. A trivial PR. I haven't added it despite it being required by RFC 1122 simply because I had no use for it.

The current TCP accept/listen mechanism, while working, seems too limited. Namely, since smolnetd has to reopen a socket every time it accepts a connection, it's impossible to connect to the host during this period. I have no concrete proposal for this yet.

It's fully equivalent to the BSD sockets. Open many sockets on the same port in the LISTEN state, that's your backlog. What smolnetd currently does is equivalent to backlog of 1.

I think ARP requests rate should be limited, it's too easy to get smoltcp flooding the network with ARP packets.

That's a bug (and a release blocker). https://github.com/m-labs/smoltcp/issues/25

I still want to see #19 implemented.

Sure, I'm looking into m-labs/smoltcp#18 now, then it'll be that. No big issues.

Currently smolnetd iterates over all the sockets after each poll to see if it should send an event/resume a blocked call. It would be handy if EthernetInterface::poll produced some kind of list of the sockets which state changed during the call.

Sounds like good functionality for SocketSet, perhaps even implemented similarly to our tracking of API consumer caused state changes.

I understand we can't free sockets in release because TCP sockets require shutdown, but maybe we should initiate the shutdown in release and then have SocketSet threat closed TCP sockets with refs == 0 as empty?

This can be implemented using the same kind of state tracking as your previous item. Socket went to closed after the process call, refs == 0, boom. We should probably just get rid of prune completely and manage reference counting in Ref entirely.

---

I'd prefer if you created a smoltcp issue for every unique item here, extracting my suggestions, and then we could continue the discussion over there.

[dlrobertson]

Can this be closed? smoltcp is the redox network stack since redox-os/netstack#4. There are still some relevant issues that @batonius mentions that are still not yet implemented, but maybe they should be split out into separate issues?

[whitequark]

smoltcp doesn't support loopback interface, an attempt to connect to 127.0.0.1 results in a flood of ARP requests asking for the MAC address of 127.0.0.1.

This is split out into #55.

smoltcp looks like a good place to implement DHCP since it already handles ARP, but I'm not sure how it should look like. Currently, Redox uses dhcpd daemon for this.

This is being handled in #75.

ICMP sockets aka IPPROTO_ICMP would be quite handy for implementing a version of ping able to work without suid, but I'm not sure if they're useful for anything besides ping. Right now Redox has icmpd daemon.

This was done in #64.

smoltcp doesn't check uniqueness of local ports when binding sockets and doesn't generate them when bind is called with port == 0

This is by design since multiple sockets on the same local port is how smoltcp implements backlog for streaming sockets. The POSIX-like behavior may be provided separately by an upper layer if necessary, I'll merge a PR implementing it but see no urgent need to add it.

It would be nice to be able to manipulate TTL of a socket since it's supported by TcpSocket and UdpSocket.

This was done in #58.

I think ARP requests rate should be limited, it's too easy to get smoltcp flooding the network with ARP packets.

This was handled in #68.

---

To summarize, yes, this issue can be closed, and everything is split out already, no need to do it.