VT sequence: file transfer

[christianparpart]

Should support:

• Chunked transmission that can be interleaved with other VT sequences (to enable displaying progress bars)
• set filename and total size in first chunk.
• Download folder cash be configured. Defaulting to ~/Downloads
• Provide script vtsend to send files
• Feature detection via DA1
• Permission check? (allow/deny/ask) - when set to deny, it should also not be advertised via DA1 or alike.

Checklist

• contour send FILE...
• contour receive FILE...

[jerch]

@christianparpart I also came across this demand, mainly from zmodem support requests. I know a few xtermjs integrations with zmodem support (and we even had an official addon for it in v3). When I took a closer look at zmodem to re-enable it for xtermjs I found it subpar for 2 reasons:

• zmodem implements its own transport protocol, which directly competes with terminal sequences and cannot be well integrated with terminal sequences (the old addon used to put an additional preparser in front of the terminal, which I find really inconvenient to deal with)
• zmodem contains tons of features for those old unreliable modem transports in the 80s, which are not really needed anymore or can be shaped more light-weighted (e.g. built-in crc-checksum with automatic resubmit of broken chunks)
• protocol is quite heavy and expensive to construct the transport bytes

That all said I think a genuine terminal sequence for file up-/download (both directions) would be good to have. And maybe we can still learn from zmodem, because it has some valuable goodies.

What I came up with conceptually so far:

• chunked transmission:
  • with strict file offset handling this can be used for incremental chunk hashing on both ends (hashing can be made optional, but will be very useful for very big files to enable resubmit on chúnk level, hash algo might be something faster than crc32)
  • allows interleaved transport (multiple transfers at once, other updates in between) - note that true "backgrounding" of file transfer is actually very tricky (would need to respect PIPE_BUF as chunksize to avoid leaking other data into the chunk data, also things like termios ECHO will make this hard, needs more investigation first)
• needs some sort of client at app side (your vtsend), but for both directions
• protocol needs some sort of handshake to await the client
• protocol needs some sort of sentinel/feedback to spot dying of the app side client (e.g. when ssh connection goes down)
• file download (client -> terminal) can be made working in any termios operation mode, if the payload maps TTY replaced chars to NOOP (like LF in base64)
• file upload (terminal -> client) is a bit more tricky in terms of termios settings - ECHO should always be disabled (otherwise the data would get mirrored back) in general ICANON mode should be off (to avoid issues with overflowing the line discipline buffer)
• Ideally a running upload would still handle interrupt user actions (needs some explicit INTBRK-like handlers on client side).
• Ideally a previously interrupted transfer can be resumed (optional, maybe mark a transfer as resume-able on the sequence for big files).
• client should work with any TE implementing the file transfer sequence (not only with contour :smile_cat:).

[christianparpart]

Oh yes. Thanks @jerch for the input. I first want to push for feedback on our unicode draft spec discussion in at least windows terminal dev community (and have it implemented as PoC such that notcurses can experiment with it). Then got image protocol (finally!). And then i think at least i have room for the file transfer idea (got image protocol outcome might come in handy here for sure).

[jerch]

@christianparpart Yes, I also think we might see some similarities between GIP and a file transfer thingy, due to the needed (prolly bigger) binary transports.