cp from pc to NodeMcu via serial messes up file system on NodeMcu

[ast0815]

Hello,

I'm having a problem with rshell and a NodeMcu ESP8266 board: Copying from the board to the pc and even editing files on the board works perfectly. But if I try to copy a file to the board over the serial connection, it just hangs and nothing happens. Then, after aborting with CTRL-C, the file system on the board is totally messed up and I have to re-flash it, to make it usable again.

Here is some of the last bit of output when I use the -d option:

Executing "cp boot.py /pyboard/"
----- About to send 458 bytes of code to the pyboard -----
def get_mode(filename):
    """Returns the mode of a file, which can be used to determine if a file
       exists, if a file is a file or a directory.
    """
    import os
    try:
        # Since this function runs remotely, it can't depend on other functions,
        # so we can't call stat_mode.
        return os.stat(filename)[0]
    except OSError:
        return 0
output = get_mode('/')
if output is None:
    print("None")
else:
    print(output)

-----
-----Response-----
b'16384\r\n'
-----
----- About to send 2124 bytes of code to the pyboard -----
def recv_file_from_host(src_file, dst_filename, filesize, dst_mode='wb'):
    """Function which runs on the pyboard. Matches up with send_file_to_remote."""
    import sys
    import ubinascii
    try:
        import pyb
        usb = pyb.USB_VCP()
        if False and usb.isconnected():
            # We don't want 0x03 bytes in the data to be interpreted as a Control-C
            # This gets reset each time the REPL runs a line, so we don't need to
            # worry about resetting it ourselves
            usb.setinterrupt(-1)
    except ImportError:
        # This means that there is no pyb module, which happens on the wipy
        pass
    try:
        with open(dst_filename, dst_mode) as dst_file:
            bytes_remaining = filesize
            if not False:
                bytes_remaining *= 2  # hexlify makes each byte into 2
            buf_size = 512
            write_buf = bytearray(buf_size)
            read_buf = bytearray(buf_size)
            while bytes_remaining > 0:
                read_size = min(bytes_remaining, buf_size)
                buf_remaining = read_size
                buf_index = 0
                while buf_remaining > 0:
                    if False:
                        bytes_read = sys.stdin.buffer.readinto(read_buf, bytes_remaining)
                    else:
                        bytes_read = sys.stdin.readinto(read_buf, bytes_remaining)
                    if bytes_read > 0:
                        write_buf[buf_index:bytes_read] = read_buf[0:bytes_read]
                        buf_index += bytes_read
                        buf_remaining -= bytes_read
                if False:
                    dst_file.write(write_buf[0:read_size])
                else:
                    dst_file.write(ubinascii.unhexlify(write_buf[0:read_size]))
                # Send back an ack as a form of flow control
                sys.stdout.write('\x06')
                bytes_remaining -= read_size
        return True
    except:
        return False
output = recv_file_from_host(None, '/boot.py', 162)
if output is None:
    print("None")
else:
    print(output)

-----
^C

The ^C is where I abort after nothing happens for a while.

Any idea what might cause this and how I can fix it?

[dhylands]

I haven't played with the esp8266 yet to see how it interacts. I think that lowering the buffer size would be worth trying. By default it picks 512 (assuming USB-serial). Lowering to something like --buffer-size=32 might help.

The file sender sends buffer-size bytes and waits for an ack. The file receiver waits for buffer-size bytes and sends an ack. There is no retry logic or timeouts, so if a character gets dropped because the serial buffers aren't big enough, then that's what causes the hang.

[ast0815]

That did the trick!

The ESP board seems to be happy with buffer sizes up to 128.