socket library: provide some sort of way to read from sockets that isn't strings

[daurnimator]

• Some protocols send a secret key over the wire.
• When reading from a socket with cqueues, the result is currently pushed as a string.
• In Lua, strings are interned, which means that depending on if the VM has seen the string before, it will take a varying amount of time to add it to the string pool.

These facts combined make many protocols subject to timing attacks if cqueues is used.

This problem can be mitigated If cqueues sockets could operate on some other non-string buffer (e.g. exposed as userdata) when given certain flags.

[daurnimator]

@fur-q brought up the idea of being able to pass lightuserdata to socket:send(). Obviously in this mode it would require the start/end offsets. (this is to be able to skip a copy from a luajit ffi pointer => lua string => cqueues)

[wahern]

There are two avenues I wanted to explore for this, not mutually exclusive:

1) fopencookie (GNU), funopen (*BSD), fmemopen (POSIX, but not widely supported). With those we could just accept a FILE handle and a length, without creating a way for reading or writing over random memory. Bindings to those APIs might not be appropriate to include with cqueues, though. But in any event I'd like cqueues.socket to directly accept FILE handles as sources/sinks.

2) Come up with a generic format specifier syntax that can be more easily extended. Currently cqueues supports the regular Lua 5.1 format plus the MIME boundary format beginning with a double-dash. On my wish list is supporting string patterns as read specifiers, which I've envisioned as beginning with "^". Adding more mechanisms like a <lightuserdata, length> vector will be messy without a proper format specifier prefix syntax.

[daurnimator]

The two options I was thinking of were:

1. exposing the fifo objects (see #64)
2. allow layering of openssl BIOs.

[wahern]

I'm all for wrapping the internal fifo object, especially because it'll make it easier to unit test it and add new features like pattern matching read formats.

I would much prefer FILE over BIO, though, and would like to see FILE regardless. One of the things about FILE is that if fileno(fp) returns a file descriptor, it can be optimized to sendfile(). So you could do things like so:write(io.open("/some/file", "r"), math.huge) for zero-copy I/O.

[daurnimator]

One of the things about FILE is that if fileno(fp) returns a file descriptor,

One of the annoyances I have with fopencookie/fmemopen/etc are that they don't allow you specify a backing fd. i.e. fileno() on a user created FILE doesn't work.

I do like the idea of sendfile here (and/or splice). but reality is that in 90% of cases you're using TLS, so it isn't useful in practice.

[wahern]

If there's a backing fd but you're doing transformations, you can't implement zero-copy and so it doesn't matter whether you can access the underlying descriptor.

If it's just a matter of polling, you can superclass socket.

What turns me off about BIO is the dependency on OpenSSL. It's probably misplaced concern, but I really like the idea of being able to pass FILE handles because it's an easier way for applications to bypass string creation without becoming more wedded to an external library. It's more limited feature wise, but it has better composability. cqueues.socket isn't the only place where it would be nice to wrap some custom data to be passed to a routine expecting a FILE handle.

Good point about TLS. Though Linux might get in-kernel TLS soon.