testcli / testsrv / testpeer need not have a console

[vanrein]

On Windows, that is a show-stopper, but it is not necessary anymore when using the chat() functionality.

[vanrein]

There's a lot more work to get this going.

[vanrein]

The chat(8) builtin module does not work. Replacing it with a large language such as Python would be rediculously large, by chat is also a bit of a nuisance. Also, it reads a character at a time, which is not useful.

We might instead use a request/response format like this:

• !<text> will write out the text (as a separate, unbuffered send() or write() operation)
• ?<regex> will match a regular expression (with a manner to bind named variables and a way to cross over line breaks)
• #<size> as a maximum to the number of bytes to follow (otherwise realloc() a buffer of, say 4096 bytes would rotate and at most 2048 bytes at a time is added while ?<regex> is tried
• @<time> will wait that many milliseconds (during which it ignores reads, or not?)
• &<time> sets a timeout in milliseconds for the next event (after which it bails out)
• *<cmd> executes a simple command, such as close gently or exit now or wait for close or errno value or error string

This language can grow with time; it can be useful to have a modern variant of chat(8) indeed!

[vanrein]

Will reopen the issue, to see if someone picks it up. Is it part of Windows porting?

[vanrein]

It may actually be fair to not rotate the buffer, since even TCP is --in practice-- message-based. The use in starttls is to bootstrap exchanges like

shell$ nc alt1.gmail-smtp-in.l.google.com 25
220 mx.google.com ESMTP r11si6374984pgm.353 - gsmtp
EHLO orvelte.nep
250-mx.google.com at your service, [83.161.146.46]
250-SIZE 157286400
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
STARTTLS 
220 2.0.0 Ready to start TLS

[vanrein]

I suppose the pattern would be this sequence of steps:

• ?^220 .*$ -- pattern match to a line starting with 220 and a space for the last response line
• !EHLO orvelte.nep\r\n -- sending our message
• ?^250-STARTTLS$ -- recognising the STARTTLS option (could be dropped if we flat-out assume its support)
• ?^250 .*$ -- accepting the closing line (this would not work if STARTTLS was on this line)
• !STARTTLS\r\n -- to initiate TLS
• ?^220 .*$ -- accept the agreement to start TLS

If the <regex> should start at the beginning of a text sent, we should prefix .* but that would be quite a style change and probably not good. The above assumes that . does not include \r or \n and that ^ and $ match inside a string with the start/end of a line. The regex(3) in libc, in the extended POSIX variant, seems to hold up with the REG_NEWLINE flag.

Update 4-4-2019: Yep, this works. Just need to do something to print \r\n properly. But at least the matches of line start/end work as predicted.

[vanrein]

Might need a either/or kind of construct; following SEQ, PAR, ALT in Occam; might add it to the commandline in (reverse) Polish notation, with parameters ;<count>, &<count>, ^<count>.

The runtime would keep in mind which elements are active when firing, they figure out how they impact each other and/or which new elements are added.

To complete a simulation of chat(8) we would also need a way to observe BREAK and perhaps other signals.

Occam also has loops. We might add an infinite loop to, with LOOP or a variant with a symbol, or leave that implicit; but + and * sound like they could do it.

[vanrein]

This might be named libpavlov and/or the pavlov command. As a library, it could be built in like we now do with chat(8).

[vanrein]

with ALT and SEQ planned, there is no need for an additional & if there's also a @ wait that can be followed by an exit command.

This is good, it means we can use & in &<count> as we accidentally did already.

[vanrein]

Implemented a few commands in 4af0d87c479d9eb9720f56b2fea9cfdd0d6d1fd5:

• ?<regex> to recognise a pattern (with ^ and $ matching per line)
• !<string> to print a literal string (with no parameter substitutions yet)
• @<msdelay> for a milli-second delay

[vanrein]

If we take this really far... which we should not do right now... then we might pickup variables from patterns and unleash them elsewhere. we might in fact store them in CSV files, and thereby allow easy collection of tables from various input formats. the ALT/SEQ/PAR would enable doing this for various tables from rather unstructured input. could be a really potent UNIX tool!

[vanrein]

• ,<filename>,<var>,<var>,<var>... may be a form to store variables (from ?<regex> matching) into a CSV file

[vanrein]

• =<var>,<var>,<var>... could be a form to name/store just-matched values from the ?<regex> form.
• easy enough to skip entries: =x,,z would fetch \0 and \2 but skip \1, for instance.

[vanrein]

@hfmanson made a variant based on libuv for; this allows testcli & co to listen to sockets and file handles such as the console's.

The basic implementation of pavlov works well, also on Windows, and can be used to replace chat(8) everywhere we now rely in it. The code was a quick and dirty insertion, but can now be dropped.

[vanrein]

PLAATJE: tlspool-tools.pdf

This is probably a good redefinition of our tools:

• testcli/srv/peer will stop to exist, at least by those names
• tlspool-chat-client/server/peer will take the role of human interaction. They will print out PRNG and Channel Binding data, to lap up by users. They will have ^Z for session resumption. Anything that feels like a textual chat session, in fact. No pavlov(3) here.
• tlsclient/server/peer will be created for automated handling. They start a subcommand, which may or may not be (or use) the pavlov(3) command or function. In general, the command has streams 0 and 1 connected to the TLS connection that it setup through the TLS Pool, so anything the command does is secure. Commands would be a bit like xinetd style daemons (which is why pavlov(3) is not a bad example). These commands are perfect for testing. They will get PRNG and Channel Binding in environment variables, for later reference or comparison or use in a SASL exchange. To mirror behaviour, the commands could maybe treat their own streams 0 and 1 as a TLS socket...? (Dunno.) It is surely possible to (also) have these command run as subcommand of (yet) another, as in upload.thingy [ tlsclient [ pavlov ?bell !drool ] ] where pavlov runs before upload.thingy. (Dunno.) The overruling concern for a test program is reliable propagation of exit() values, so let's not do these frivolous things. (Yet.)
• tlstunnel continues to exist as a scalable server that churns away in the background, connecting massive incoming plaintext sessions over TLS with possible pavlov(3) initiation to get beyond the point of STARTTLS agreement. It can do the opposite thing on the other end.

[vanrein]

resolved (mostly) in e9c832b180d553bb12b8078977a975c11b841e5a

[vanrein]

The construct of ALT with a branch of SEQ of an @time and exit is useful for Windows, where this precise construct is now hacked into Pavlov to accommodate timeouts of the tools.