Decide on pipe protocol

[michaeljklein]

So we've decided to use posix pipes to call/return from the recompiled C code. Great, as they're often very fast and should port to Linux without too much difficulty. Now we need to figure out what kind of protocol to use to pass back and forth.

Notes:

• Need to be able to pass all standard (non-pointer) C types, plus structs
• Pipes function much like file IO, and so only support char as an IO type
• Currently available protocols are likely too powerful for this use case as the only variables relevant to formatting (after the C code is generated) are 1) optional arguments (need a "these args are being sent, these are not" arg), 2) array length
• All types are known by both sides, so don't need to worry about saying what is what, just sending in a consistent order
• Converting an arbitrary fixed-size non-pointer type is easy enough, just pull out 8 bits however many times
• For arrays, one possibility is to send the first SIZE_MAX / 8 chars as the size of the array (on my computer, a 2011 Macbook Pro (64-bit), SIZE_MAX is 18446744073709551615ULL, 2^64 - 1, or 8 chars. The thing is though, one wouldn't expect the generated program to be piped that much; it's just not really in the scope of the project. Then, maybe consider having a defined end. However, the branch to check for the end + any conversion from the "natural" type to one that leaves room for an end number could be a lot of trouble to implement and its branching could take as long as piping 8 chars.
• Will the runner be able to read from the pipe as soon as it starts running? Does it need a handshake with the caller to initialize? Does the reader need to be listening exactly when the writer is sending? How does the caller know when the runner has finished?

[michaeljklein]

1) I've decided to use a C import for the readers/writers 2) I'm using named pipes so mkfifo() 3) Reading/writing fixed-width "units" is being acheived by splitting into chars and then using putc/getc

[michaeljklein]

[15:30] MichaelK: why not just fork+exec? [15:31] o11c: that's brilliant. [15:35] o11c: just to check, the flow would be something like: local variable defined in main, path of newly compiled file new.c passed to main, main forks, main calls exec() to overwrite process with new.c, new process calls function on local variable defined in main, main can access results? [15:36] ah hm, if you want to get anything beyond a single byte that wouldn't work too well [15:36] MichaelK: actually, have you considered dlopen() if you trust the fresh code? [15:37] o11c: no, unfortunately I haven't googles [15:37] o11c: I do trust the code, as the program is generating it [15:41] o11c: I'm just floored that it's that easy (and unsafe, of course). [15:41] for more fun, there is a new interface, dlmopen, that lets you maintain entirely separate and incompatible sets of lbiraries [15:44] o11c: that also amazes me, but it doesn't look like I'd need it. (unless you think it'd be better practice to have the extra info or it'd help significantly with testing) [15:51] MichaelK: nah, you don't need it, it's just cool to keep these things in mind

So yeah, this is completely blown out of the water with how much it's solved.