tapdevice.c can't be built on not linux platforms

[cartazio]

looks like its doing a raw socket thing.... should be possible to implement in a more portable way, but will need to investigate further

[ghost]

Oops. Commented on issue 1 for this. I've sent a pull request that should make it build on OS X.

[cartazio]

cool, i'll check it out later today or tomororw, thanks! (also: would it perhaps make sense to see about replacing the cbits stuff currently there with just using raw sockets?)

[ghost]

We should really try and use the underlying OS socket mechanisms. The TUN/TAP devices are good for some things, but they'll need a lot more configuration on the host for us to actually communicate with the world. And I'm a bit surprised Haskell doesn't have some sort of IO monad for read/write/send/recv already, but I also wonder if this package was originally something different than what we're looking for it to do.

[cartazio]

Yeah, HaNS was meant for doing bare metal on top of XEN, so that might account for the funny bizness there

do you mean like the sockets api here? http://hackage.haskell.org/package/network (using raw sockets perhaps?)

Or do you have another idea in mind?

Either way, i agree, if we want the goal to have a portal userland networking stack with minimal c code, theres some better ways that the preexisting code does things

(and thats also ignoring how to organize things for good performance wrt the tradeoff space of throughput, latency, reliability, or at least look into such)

[ghost]

Oooh, yeah, maybe that. I'm going to have to dig in and see. And if we can be really minimal, and just use basic BSD SOCK_RAW (http://sock-raw.org/papers/sock_raw) with send/recv (and I can't see why this wouldn't work from Haskell itself unless there's no way to do system calls at all in IO), we should be able to get by without C at all. I'll dig deeper and see what I can find out.

[cartazio]

either way, the core sys calls do an ffi call to the associated c-libs, just like everyone else.

I think that the System.Network approach is pretty much that (plus extra bits).

Alternatively, we coud do a minimal thing that replicates just the raw socket subset of the ffi there, with the right flag settings per OS so that we have uniform networking semantics?

[ghost]

Well, yes, of course syscalls do FFI calls, but if we can use an existing mechanism for that which is already in Haskell or the standard library we don't have to write our own and we can work on the network stuff more. But I do plan on looking at what the raw sockets in Network do, and whatever else I can find, and see what's out there. Or I could just forge ahead and do some FFI stuff and move on, too...

[ghost]

We need something like System.Posix.Sockets or something (http://www.haskell.org/ghc/docs/6.12.2/html/libraries/unix-2.4.0.1/System-Posix.html). Maybe that's the place to start. See how they handle read/write, and then do socket/send/recv there and see what happens.

[cartazio]

http://hackage.haskell.org/package/network-2.4.0.1 is the current version of the lib to look at :)

yeah, i'd favor sorting out swapping the cbits stuff to using the Network.Socket and some (combination?) of Network.Socket.ByteString and/or Network.Socket.LazyByteString.

what i may do today or by the close of this weekend have a go at trying to do an OSX only udp via raw sockets implementation or something like that. Seems like the sanest simple way to test things out in terms of performance and the like.... or at least sounds like such myself

another interesting thing to consider is then giving a push/pull streaming layer over the underlying imperative chunks of stuff, perhaps using the new pipes library http://hackage.haskell.org/package/pipes-3.0.0, would substantially improve the engineering story and overall not hinder having nice performance.

[ghost]

I'm looking at Network now. It does seem to do exactly what I was describing after a fashion. Finding a way to reuse that or copy it would probably be the way to go I think. And maybe if we do it the same way other Haskell packages could easily integrate too. And we could even add SCTP easily enough or something esoteric and kinky like that...

[cartazio]

Yes, so theres 2 ways we could go about it: we could

a) be "ironic" and just have the network package be dependency (but that would be outright circuitous) , which would be lame :-(

b) have a generic notion of "backend" with a fixed simple set of generic prim ops, and provide 1 based upon the network package, maybe one based upon a more cross platform of the cbits code + haskell layer on top, and maybe some other ones (like our own minimal teeny bit of see to capture the raw sockets bits we care about and igore the rest?)

basically: either way we should figure out what the "minimal" / "core" raw sockets api in the IO monad for haskell mades sense, with some sort of support for both ByteString and LazyByteString versions, and use that api for the rest!

[cartazio]

a good example of approach (b) is the diagrams library, which has a pretty rich (relative to what we might be initially doing) surface area of api / stuff that needs to be mapped to a backend

[cartazio]

theres also the lack of ipv6 support (and that seems important)

a bunch of these ideas / subtasks I'll spin out as a bunch of tickets