`Screen.report_device_attributes` with xterm

[acroz]

I've been working on adapting the webterm example to work with our terminado replacement, and am encountering an issue when using it with xterm. Below is a Python example which adapts the webterm example to just run vim and display the output:

import os
import pty
import shlex
import signal
import time

import pyte

class Terminal:
    def __init__(self, columns, lines, p_in):
        self.p_in = p_in
        self.screen = pyte.HistoryScreen(columns, lines)
        self.screen.set_mode(pyte.modes.LNM)
        self.screen.write_process_input = \
            lambda data: p_in.write(data.encode())
        self.stream = pyte.ByteStream()
        self.stream.attach(self.screen)

    def feed(self, data):
        self.stream.feed(data)

    def print(self):
        line_length = len(self.screen.display[0])
        print('+' + ('-' * line_length) + '+')
        for line in self.screen.display:
            print('|{}|'.format(line))
        print('+' + ('-' * line_length) + '+')

def open_terminal(command="bash", columns=40, lines=12):
    p_pid, master_fd = pty.fork()
    if p_pid == 0:  # Child.
        argv = shlex.split(command)
        env = dict(TERM="xterm", LC_ALL="en_GB.UTF-8",
                   COLUMNS=str(columns), LINES=str(lines))
        os.execvpe(argv[0], argv, env)

    # File-like object for I/O with the child process aka command.
    p_out = os.fdopen(master_fd, "w+b", 0)
    return Terminal(columns, lines, p_out), p_pid, p_out

if __name__ == "__main__":

    terminal, p_pid, p_out = open_terminal()

    time.sleep(0.1)
    p_out.write(b'vim\n')
    time.sleep(0.1)

    print('Before `write_process_input` has effect:')
    terminal.feed(p_out.read(65536))
    terminal.print()

    print('After `write_process_input` has effect:')
    terminal.feed(p_out.read(65536))
    terminal.print()

    os.kill(p_pid, signal.SIGTERM)
    p_out.close()

This generates the following output:

Before `write_process_input` has effect:
+----------------------------------------+
|                                        |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|                                        |
+----------------------------------------+
After `write_process_input` has effect:
+----------------------------------------+
|<9b>?6c                                 |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|~                                       |
|-- REPLACE --                           |
+----------------------------------------+

As you can see, some control characters get written into vim, causing it to enter replace mode and enter some text. These characters are written by pyte, which catches some control characters coming from the terminal output and calls Screen.report_device_attributes, which in turn sends CSI + "?6c" back into the terminal via the write_process_input method.

Comments in the code reference Linux's VT102 terminal emulation code, and indeed if I set TERM='vt102', the problem goes away. Might it be the case that this code should only have an effect when the terminal process the screen is attached to is in a particular mode?

For the moment I am avoiding this issue by simply removing the attachment of write_process_input back onto the terminal process, but it would be good if anyone can provide feedback on whether this is the right thing to do.

Thanks, Andrew

[acroz]

After further investigation, I found that the culprit seems to be that the Control Sequence Introducer sent back is '\x9b' instead of '\x1b[' (ESC + [). This Wikipedia paragraph suggests that for interoperability with Unicode, using the 7-bit compatible C0 sequence '\x1b[' will be more robust than the single character C1 code.

If I change CSI in pyte.control to '\x1b[', my problem above goes away - both the device attributes and device status requests are sent back into the terminal successfully.

[acroz]

The above PR patches pyte to avoid this issue.