`mpremote`: Mounting Windows directory with packages in a `lib` subfolder are not found

[jerrens]

If I connect to my pico device and mount my project directory (in Windows 10), I get an ImportError: no module named '...' when trying to import a file that is inside a lib subfolder. However, if I include \lib to sys.path so that it expects the Windows directory separator, then things will work as expected.

Is there some other recommended way to do this when developing in Windows? I'm raising this as a bug report because I expected that the mount feature in mpremote would automatically accommodate this directory separator, or that MicroPython's sys package would be able to adapt. Adding \lib to the path doesn't seem like correct approach. If I were to use Thonny to upload all files to the pico, then it would be in the correct /lib folder, so when deploying the project (after development), I'll need to change this logic in the code.

I've done a little work in Python in the past, but I'm very new to MicroPython itself, so I'm hoping I'm missing something obvious. If others do not consider this to be a bug, then I'll close and move it over to Discussions for advice if that is more appropriate.

Example:

C:\Projects\W5500-EVB-Pico\demo>mpremote connect port:COM9 mount .
Local directory . is mounted at /remote
Connected to MicroPython at COM9
Use Ctrl-] to exit this shell
>
MicroPython bdbc444 on 2022-09-27; W5500-EVB-Pico with RP2040
Type "help()" for more information.
>>> import sys
>>> sys.path
['', '.frozen', '/lib']
>>> import queue
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: no module named 'queue'
>>>
>>>
>>> sys.path.insert(2, '\lib')
>>> sys.path
['', '.frozen', '\\lib', '/lib']
>>> import queue
>>>

Ultimately I'm trying to create a task for VS Code (.vscode\task.json) that will mount my project directory and automatically run main.py. I thought I had things working, but realized that my changes to a file under the lib folder in my project folder (on PC) were not showing up when running. It turns out it was running an old copy of my code that I had manually uploaded (to the lib folder in the pico VFS) instead of the version in the mount point since the sys.path was looking in /lib (local) instead of \lib (mount).

My solution for now is to:

1. Add the following code at the beginning of main.py

   import sys
   sys.path.insert(2, '\lib') # Look in the mounted 'lib' directory first, before the local 'lib' directory

2. Run the command: mpremote connect port:COM9 mount . run main.py from the console where the CWD is my project folder.

Other details:

My project folder has the following contents:

C:\Projects\W5500-EVB-Pico\demo
|   .gitignore
|   cli.py
|   demo.code-workspace
|   fileops.py
|   log.txt
|   main.py
|   README.md
|
+---.vscode
|       tasks.json
|
+---api
|       testRoutes.py
|
+---html
|       flag.html
|
+---lib
|   |   queue.py
|   |
|   \---phew
|           dns.py
|           logging.py
|           ntp.py
|           server.py
|           template.py
|           __init__.py
|
\---tests
    \---RESTClient
            flag.http

[jimmo]

When the remote filesystem is mounted, it is mounted in /remote. So the entry that needs to go in sys.path is "/remote/lib".

[jerrens]

I was able to change sys.path to:

import sys
sys.path[2] = 'lib' # Replace the default `/lib` entry with just `lib` to remove the incorrect directory separator when mounted to a Windows filesystem

instead and this fixed the problem I was having.

What impact would this have when mounting other filesystems (other than Windows NTFS)?

[jerrens]

After the discussion on https://github.com/micropython/micropython/issues/9735 with @jimmo, my plan is to change my sys.path to be: ['', '.frozen', 'lib', '/lib'] instead. This way, when mounting via mpremote ... mount . (current working directory becomes /remote), then contents of the relative lib folder will have a higher priority than the system /lib folder.

If the consensus is that this is a custom fix for my scenario and not anything that should be changed in MicroPython or mpremote, then this issue can be closed.