ls /pyboard with rshell fails

[marcoschulte]

Hi everyone, I have a problem with rshell. Though I do know that you most certainly won't be able to solve it, I am hoping you might have an idea who I could contact to solve this, as you seem far more experienced hacking around with mindstorms.

I just got the new mindstorms and started poking around with it. I have no experience with micropython and read about rshell here for the first time. Unfortunately I have a pretty fundamental problem. Whenever I try to list the /pyboard directory using rshell, an error is thrown, as the result cannot be parsed. It seems like the result from the helper functions print is incomplete and the beginning is missing. I already noticed this behavior a couple of days before when I was just using screen to connect to the mindstorms and started printing the content of files out of curiosity whats going on on the mindstorms: In my helper functions I had to read the files line by line and use separate print commands for each line to get the complete file content. Otherwise I had the same behavior that I was just seeing the end of the files. It seems like the mindstorms print command has some hiccups, too small buffer, I don't know.. Do you have any idea who to contact to somehow solve this?

In case you are interested, this is how the debug output of rshell looks like

/Users/mschulte> ls /pyboard/
Executing "ls /pyboard/"
----- About to send 634 bytes of code to the pyboard -----
def stat(filename):
    """Returns os.stat for a given file, adjusting the timestamps as appropriate."""
    import os
    try:
        # on the host, lstat won't try to follow symlinks
        rstat = os.lstat(filename)
    except:
        rstat = os.stat(filename)
    return rstat[:7] + tuple(tim + 946681200 for tim in rstat[7:])

def get_stat(filename):
    """Returns the stat array for a given file. Returns all 0's if the file
       doesn't exist.
    """
    try:
        return stat(filename)
    except OSError:
        return (0,) * 10
output = get_stat('/')
if output is None:
    print("None")
else:
    print(output)

-----
-----Response-----
b'(16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)\r\n'
-----
----- About to send 1207 bytes of code to the pyboard -----
def is_visible(filename):
    """Determines if the file should be considered to be a non-hidden file."""
    return filename[0] != '.' and filename[-1] != '~'

def stat(filename):
    """Returns os.stat for a given file, adjusting the timestamps as appropriate."""
    import os
    try:
        # on the host, lstat won't try to follow symlinks
        rstat = os.lstat(filename)
    except:
        rstat = os.stat(filename)
    return rstat[:7] + tuple(tim + 946681200 for tim in rstat[7:])

def listdir_stat(dirname, show_hidden=True):
    """Returns a list of tuples for each file contained in the named
       directory, or None if the directory does not exist. Each tuple
       contains the filename, followed by the tuple returned by
       calling os.stat on the filename.
    """
    import os
    try:
        files = os.listdir(dirname)
    except OSError:
        return None
    if dirname == '/':
        return list((file, stat('/' + file)) for file in files if is_visible(file) or show_hidden)
    return list((file, stat(dirname + '/' + file)) for file in files if is_visible(file) or show_hidden)
output = listdir_stat('/')
if output is None:
    print("None")
else:
    print(output)

-----
-----Response-----
b"00, 946681200)), ('sounds', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('spike', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('ui', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('hub_runtime.mpy', (32768, 0, 0, 0, 0, 0, 2015, 946681200, 946681200, 946681200)), ('version.py', (32768, 0, 0, 0, 0, 0, 54, 946681200, 946681200, 946681200)), ('.runtime_hash', (32768, 0, 0, 0, 0, 0, 953, 946681200, 946681200, 946681200)), ('extra_files', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('.extra_files_hash', (32768, 0, 0, 0, 0, 0, 539, 946681200, 946681200, 946681200))]\r\n"
-----
Traceback (most recent call last):
  File "/usr/local/bin/rshell", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/rshell/command_line.py", line 4, in main
    rshell.main.main()
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 2966, in main
    real_main()
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 2949, in real_main
    shell.cmdloop(cmd_line)
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 1755, in cmdloop
    cmd.Cmd.cmdloop(self)
  File "/usr/local/Cellar/python@3.9/3.9.1_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/cmd.py", line 138, in cmdloop
    stop = self.onecmd(line)
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 1785, in onecmd
    self.onecmd_exec("".join(group))
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 1796, in onecmd_exec
    return cmd.Cmd.onecmd(self, line)
  File "/usr/local/Cellar/python@3.9/3.9.1_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/cmd.py", line 217, in onecmd
    return func(arg)
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 2423, in do_ls
    ldir_stat = auto(listdir_stat, filename)
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 588, in auto
    return dev.remote_eval(func, dev_filename, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rshell/main.py", line 1515, in remote_eval
    return eval(self.remote(func, *args, **kwargs))
  File "<string>", line 1
    00, 946681200)), ('sounds', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('spike', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('ui', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('hub_runtime.mpy', (32768, 0, 0, 0, 0, 0, 2015, 946681200, 946681200, 946681200)), ('version.py', (32768, 0, 0, 0, 0, 0, 54, 946681200, 946681200, 946681200)), ('.runtime_hash', (32768, 0, 0, 0, 0, 0, 953, 946681200, 946681200, 946681200)), ('extra_files', (16384, 0, 0, 0, 0, 0, 0, 946681200, 946681200, 946681200)), ('.extra_files_hash', (32768, 0, 0, 0, 0, 0, 539, 946681200, 946681200, 946681200))]
                 ^
SyntaxError: unmatched ')'

[gpdaniels]

As a work around for now you can use rshell to dump the filesystem to your local computer and then look at the structure there. See the directories in the filesystem directory for versions I've already dumped.