File upload is broken on ESP8266 on newer Micropython versions

[j0ju]

Starting with Micropython/master @ f5f4cdae89ed040ae9209a380cf968254434e819 the file upload does not work anymore on ESP8266 devices. Other devices might be affected, too. I nailed it down by bisecting.

How to reproduce: Check out Micropython master with at least the mentioned commit. Compile it Flash it Try to transfer a file

( echo "open ttyUSB0"; echo "put boot.py" ) | mpfshell --nocolor --nocache

With a working version f5f4cdae89ed040ae9209a380cf968254434e819~1 it will work.

With later version you get this:

$ ( echo "open ttyUSB0"; echo "put boot.py" ) | mpfshell --nocolor --nocache

Micropython File Shell v0.7.6, sw@kaltpost.de -- Running on Python 3.4 using PySerial 3.2.1 --

mpfs [/]> Connected to esp8266 mpfs [/]> Traceback (most recent call last): File "/home/abra/opt/python3-venv/bin/mpfshell", line 8, in main() File "/home/abra/opt/python3-venv/lib/python3.4/site-packages/mp/mpfshell.py", line 687, in main mpfs.cmdloop() File "/usr/lib/python3.4/cmd.py", line 138, in cmdloop stop = self.onecmd(line) File "/usr/lib/python3.4/cmd.py", line 217, in onecmd < return func(arg) File "/home/abra/opt/python3-venv/lib/python3.4/site-packages/mp/mpfshell.py", line 365, in do_put self.fe.put(lfile_name, rfile_name) File "/home/abra/opt/python3-venv/lib/python3.4/site-packages/mp/retry.py", line 43, in fretry return f(*args, **kwargs) File "/home/abra/opt/python3-venv/lib/python3.4/site-packages/mp/mpfexp.py", line 277, in put raise e File "/home/abra/opt/python3-venv/lib/python3.4/site-packages/mp/mpfexp.py", line 259, in put self.exec("f = open('%s', 'wb')" % self.fqn(dst)) File "/home/abra/opt/python3-venv/lib/python3.4/site-packages/mp/pyboard.py", line 157, in exec raise PyboardError('exception', ret, ret_err) mp.pyboard.PyboardError: ('exception', b'', b'Traceback (most recent call last):\r\n File "", line 1, in >\r\nOSError: [Errno 19] ENODEV\r\n')

[mhoffma]

yes the newer version of micropython moved its root I added a patch to this repo to fix it has been posted. Basically vfat stuff has been added to esp port and the flash is now mounted /flash and / is not writeable I believe which causes problems, to fix this I just do a os.getcwd() to determine where to work from this maintains compatibility between both versions of the kernel pre /flash.

https://github.com/wendlers/mpfshell/pull/22

you could pull from here and try

https://github.com/mhoffma/mpfshell

Let me know if something it doesn't work.

[j0ju]

Your commit indeed does fix the situation. TY.

[kfricke]

A PR for this main repository would be nice!

[mhoffma]

https://github.com/wendlers/mpfshell/pull/22

[xantoz]

I independently fixed this on my own (plus a few other problems), before I realized somebody else had also attacked the problem... PR is #27

[mhanuel26]

I believe the issue is not fixed with last micropython, today I install last esp8266 build from source, when listing directory after connecting I got

Connected to esp8266 mpfs [/]> ls Traceback (most recent call last): File "/usr/local/bin/mpfshell", line 8, in main() File "/usr/local/lib/python2.7/dist-packages/mp/mpfshell.py", line 687, in main mpfs.cmdloop() File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop stop = self.onecmd(line) File "/usr/lib/python2.7/cmd.py", line 221, in onecmd return func(arg) File "/usr/local/lib/python2.7/dist-packages/mp/mpfshell.py", line 205, in do_ls files = self.fe.ls(add_details=True) File "/usr/local/lib/python2.7/dist-packages/mp/mpfexp.py", line 503, in ls files = MpFileExplorer.ls(self, add_files, add_dirs, add_details) File "/usr/local/lib/python2.7/dist-packages/mp/retry.py", line 43, in f_retry return f(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/mp/mpfexp.py", line 195, in ls raise e mp.pyboard.PyboardError: ('exception', '', 'Traceback (most recent call last):\r\n File "", line 1, in \r\nOSError: [Errno 19] ENODEV\r\n') Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/usr/local/lib/python2.7/dist-packages/mp/conwebsock.py", line 69, in run self.ws.run_forever() File "/usr/local/lib/python2.7/dist-packages/websocket/_app.py", line 242, in run_forever close_frame.data if close_frame else None) File "/usr/local/lib/python2.7/dist-packages/websocket/_app.py", line 249, in _get_close_args import inspect RuntimeError: sys.path must be a list of directory names

I tried xantoz as well as mhoffma forks whitout success.

I will appreciate any comments,

[xantoz]

I'm unable to reproduce the above running my branch of mpfshell on either python2.7 or python3.4 on latest micropython master (esp8266 port) as of writing this message (0982884655a2b8619f8334b5fdb9f2e1f59670b4). Also, on my branch (and probably also mhoffmas, but I don't know if it updates the prompt on connection like I do) I would expect it to start in "/flash", not "/"

I connected over UART. This was on a freshly flashed board, websockets off. I should probably try over websockets too (later, when I have time), as that seems to figure in your trace.

[mhanuel26]

Hello xantoz,

Same with USB port,

mpfs [/]> open ttyUSB2 Connected to esp8266 mpfs [/]> ls Traceback (most recent call last): File "/usr/local/bin/mpfshell", line 8, in main() File "/usr/local/lib/python2.7/dist-packages/mp/mpfshell.py", line 687, in main mpfs.cmdloop() File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop stop = self.onecmd(line) File "/usr/lib/python2.7/cmd.py", line 221, in onecmd return func(arg) File "/usr/local/lib/python2.7/dist-packages/mp/mpfshell.py", line 205, in do_ls files = self.fe.ls(add_details=True) File "/usr/local/lib/python2.7/dist-packages/mp/mpfexp.py", line 503, in ls files = MpFileExplorer.ls(self, add_files, add_dirs, add_details) File "/usr/local/lib/python2.7/dist-packages/mp/retry.py", line 43, in f_retry return f(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/mp/mpfexp.py", line 195, in ls raise e mp.pyboard.PyboardError: ('exception', '', 'Traceback (most recent call last):\r\n File "", line 1, in \r\nOSError: [Errno 19] ENODEV\r\n')

I was working with an old version of micropython without the issue. After I merge my fork with last master error start. Here is my Fork , I only have some modifications for enable/disable the serial console but those mods were working with mpfshell before the merge.

Please let me know if you see something I might change to test.

[mhanuel26]

I confirm with the latest image (512 version) that mpfshell has the issue.

MicroPython v1.8.7-334-g1f549a3 on 2017-03-02; ESP module with ESP8266 Type "help()" for more information.

mpfs [/]> open ttyUSB2 Connected to esp8266 mpfs [/]> ls Traceback (most recent call last): File "/usr/local/bin/mpfshell", line 8, in <module> main() File "/usr/local/lib/python2.7/dist-packages/mp/mpfshell.py", line 687, in main mpfs.cmdloop() File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop stop = self.onecmd(line) File "/usr/lib/python2.7/cmd.py", line 221, in onecmd return func(arg) File "/usr/local/lib/python2.7/dist-packages/mp/mpfshell.py", line 205, in do_ls files = self.fe.ls(add_details=True) File "/usr/local/lib/python2.7/dist-packages/mp/mpfexp.py", line 503, in ls files = MpFileExplorer.ls(self, add_files, add_dirs, add_details) File "/usr/local/lib/python2.7/dist-packages/mp/retry.py", line 43, in f_retry return f(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/mp/mpfexp.py", line 169, in ls res = self.eval("os.listdir('%s')" % self.dir) File "/usr/local/lib/python2.7/dist-packages/mp/pyboard.py", line 150, in eval ret = self.exec_('print({})'.format(expression)) File "/usr/local/lib/python2.7/dist-packages/mp/pyboard.py", line 157, in exec_ raise PyboardError('exception', ret, ret_err) mp.pyboard.PyboardError: ('exception', '', 'Traceback (most recent call last):\r\n File "<stdin>", line 1, in <module>\r\nAttributeError: \'module\' object has no attribute \'listdir\'\r\n')

[wendlers]

merged in the PRs mentioned which sould fix the problems.