Closed trepidacious closed 1 year ago
@trepidacious Thanks for the detailed report :smile:
I will take a deeper look into this this weekend, but one thing I'm curious about is whether you see the same behavior when using the rshell backend (via micropy stubs create --backend rshell ...
)
That would help narrow down where the root of the issue is (stub repo/management or pyb with pydevice backend, otherwise).
@BradenM You're very welcome, thanks for the really useful tool! :)
I've tried running with that option but I get an error, not sure whether I'm using the right params?
$micropy stubs create --backend rshell /dev/cu.usbmodem11201
Usage: micropy stubs create [OPTIONS] PORT
Try 'micropy stubs create --help' for help.
Error: No such option: --backend
I can only see --help
and --verbose
options when I run micropy stubs create --help
.
rshell
is installed and works with the board on the same serial device.
One thing I forgot to mention - following the docs for creating stubs I ran pip install micropy-cli[create_stubs]
(actually I had to use pip install "micropy-cli[create_stubs]"
on zsh), but this gave:
Requirement already satisfied: micropy-cli[create_stubs] in /opt/homebrew/lib/python3.10/site-packages (4.1.0)
WARNING: micropy-cli 4.1.0 does not provide the extra 'create_stubs'
...
Not sure whether that means I'm missing a dependency?
@trepidacious :facepalm: I seem to have gotten a bit ahead of myself… the functionality I was referring to is not released yet and is still in a PR https://github.com/BradenM/micropy-cli/pull/394
:upside_down_face: Sorry about that.
That said, it looks as if I may have already solved this issue in #394: https://github.com/BradenM/micropy-cli/pull/394/files#diff-defa6605073200674e421571ce0f08acdde68675dd1c5e97237c360073c3b741L143-R152
I intend on pushing out a prelease once I get it merged (or at most soon after). I will leave this issue open for now so others may find your workaround if needed.
One thing I forgot to mention - following the docs for creating stubs I ran pip install micropy-cli[create_stubs] (actually I had to use pip install "micropy-cli[create_stubs]" on zsh), but this gave
This is expected. The create_stubs
extra was removed a while back -- I have just been slow to update the documentation :grimacing: (#395)
@BradenM Ah that's great - I'll stand by for the prerelease, thanks!
@trepidacious https://github.com/BradenM/micropy-cli/releases/tag/v4.2.0-beta is out -- please let me know if you find the time to test it out and can confirm this issue can be closed Thanks!
@BradenM Thanks, I've had a look and unfortunately I still get an error, it seems to be different now though:
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/network.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/ntptime.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/onewire.py: | | [0.00/0.00 @ ?B/s]
MicroPy Copied Stubs: stubs
Traceback (most recent call last):
File "/opt/homebrew/bin/micropy", line 8, in <module>
sys.exit(cli())
File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/homebrew/lib/python3.10/site-packages/click/decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/homebrew/lib/python3.10/site-packages/micropy/cli.py", line 314, in create
return mpy.create_stubs(port, verbose=verbose, backend=backend)
File "/opt/homebrew/lib/python3.10/site-packages/micropy/main.py", line 151, in create_stubs
stub_path = self.stubs.from_stubber(stub_path, out_dir)
File "/opt/homebrew/lib/python3.10/site-packages/micropy/stubs/stubs.py", line 355, in from_stubber
mod_file = next(_path.rglob("modules.json"))
StopIteration
I used pip install --upgrade --pre micropy-cli
to upgrade, micropy --version
gives micropy, version 4.2.0b0
I've repeated a few times with the same results - it looks like nothing is produced in ~/.micropy/stubs
micropy.log
shows:
[2023-03-22 20:05:55] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function PackagesModule.add_from_file at 0x10569b7f0>, '_hook__add_from_file__path_None__dev_False')
[2023-03-22 20:05:55] DEBUG: hookproxy(add_package): Method added to proxy: (<function PackagesModule.add_package at 0x10569b910>, '_hook__add_package__dev_False')
[2023-03-22 20:05:55] DEBUG: hookproxy(add_package): Method added to proxy: (<function DevPackagesModule.add_package at 0x10569bd90>, '_hook__add_package__dev_True')
[2023-03-22 20:05:55] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function DevPackagesModule.add_from_file at 0x10569beb0>, '_hook__add_from_file__path_None__dev_True')
[2023-03-22 20:05:55] DEBUG: hookproxy(stubs): Method added to proxy: (<function StubsModule.stubs at 0x107ec31c0>, '_hook__stubs__')
[2023-03-22 20:05:55] DEBUG: hookproxy(add_stub): Method added to proxy: (<function StubsModule.add_stub at 0x107ec3640>, '_hook__add_stub__')
[2023-03-22 20:05:55] DEBUG: micropy: MicroPy Loaded
[2023-03-22 20:05:55] DEBUG: micropy: Running first time setup...
[2023-03-22 20:05:55] DEBUG: micropy: Creating .micropy directory @ /Users/rebeam/.micropy
[2023-03-22 20:05:55] INFO: micropy: Connecting to Pyboard @ /dev/cu.usbmodem21201
[2023-03-22 20:05:55] DEBUG: pyboard: Starting new HTTPS connection (1): raw.githubusercontent.com:443
[2023-03-22 20:05:55] DEBUG: pyboard: Starting new HTTPS connection (1): raw.githubusercontent.com:443
[2023-03-22 20:05:55] DEBUG: pyboard: https://raw.githubusercontent.com:443 "GET /BradenM/micropy-stubs/master/source.json HTTP/1.1" 200 1535
[2023-03-22 20:05:55] DEBUG: pyboard: https://raw.githubusercontent.com:443 "GET /Josverl/micropython-stubs/main/publish/package_data.jsondb HTTP/1.1" 200 4053
[2023-03-22 20:05:56] INFO: micropy: Connected!
[2023-03-22 20:05:56] INFO: micropy: Executing stubber on pyboard...
[2023-03-22 20:06:05] INFO: micropy: Done!
[2023-03-22 20:06:05] INFO: micropy: Copying stubs...
[2023-03-22 20:08:50] INFO: micropy: Copied Stubs: stubs
[2023-03-23 09:00:26] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function PackagesModule.add_from_file at 0x105c437f0>, '_hook__add_from_file__path_None__dev_False')
[2023-03-23 09:00:26] DEBUG: hookproxy(add_package): Method added to proxy: (<function PackagesModule.add_package at 0x105c43910>, '_hook__add_package__dev_False')
[2023-03-23 09:00:26] DEBUG: hookproxy(add_package): Method added to proxy: (<function DevPackagesModule.add_package at 0x105c43d90>, '_hook__add_package__dev_True')
[2023-03-23 09:00:26] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function DevPackagesModule.add_from_file at 0x105c43eb0>, '_hook__add_from_file__path_None__dev_True')
[2023-03-23 09:00:26] DEBUG: hookproxy(stubs): Method added to proxy: (<function StubsModule.stubs at 0x10846b1c0>, '_hook__stubs__')
[2023-03-23 09:00:26] DEBUG: hookproxy(add_stub): Method added to proxy: (<function StubsModule.add_stub at 0x10846b640>, '_hook__add_stub__')
Just realised I'd chopped off a bit too much - there's also a "Failed to create the report" message earlier in the output:
MicroPy Pyboard Stub module: urandom to file: /stubs/micropython-v1_19_1-rp2/urandom.py mem:116656
MicroPy Pyboard Stub module: ure to file: /stubs/micropython-v1_19_1-rp2/ure.py mem:116656
MicroPy Pyboard Stub module: urequests to file: /stubs/micropython-v1_19_1-rp2/urequests.py mem:116656
MicroPy Pyboard Stub module: urllib/urequest to file: /stubs/micropython-v1_19_1-rp2/urllib/urequest.py Skip module: urllib.urequest Module not found.
MicroPy Pyboard Stub module: uselect to file: /stubs/micropython-v1_19_1-rp2/uselect.py mem:116640
MicroPy Pyboard Stub module: usocket to file: /stubs/micropython-v1_19_1-rp2/usocket.py mem:116640
MicroPy Pyboard Stub module: ussl to file: /stubs/micropython-v1_19_1-rp2/ussl.py mem:116640
MicroPy Pyboard Stub module: ustruct to file: /stubs/micropython-v1_19_1-rp2/ustruct.py mem:116640
MicroPy Pyboard Stub module: usys to file: /stubs/micropython-v1_19_1-rp2/usys.py mem:116640
MicroPy Pyboard Stub module: utelnetserver to file: /stubs/micropython-v1_19_1-rp2/utelnetserver.py Skip module: utelnetserver Module not found.
MicroPy Pyboard Stub module: utime to file: /stubs/micropython-v1_19_1-rp2/utime.py mem:116624
MicroPy Pyboard Stub module: utimeq to file: /stubs/micropython-v1_19_1-rp2/utimeq.py Skip module: utimeq Module not found.
MicroPy Pyboard Stub module: uwebsocket to file: /stubs/micropython-v1_19_1-rp2/uwebsocket.py mem:116608
MicroPy Pyboard Stub module: uzlib to file: /stubs/micropython-v1_19_1-rp2/uzlib.py mem:116608
MicroPy Pyboard Stub module: version to file: /stubs/micropython-v1_19_1-rp2/version.py mem:116608
MicroPy Pyboard Stub module: websocket to file: /stubs/micropython-v1_19_1-rp2/websocket.py mem:116528
MicroPy Pyboard Stub module: websocket_helper to file: /stubs/micropython-v1_19_1-rp2/websocket_helper.py Skip module: websocket_helper Module not found.
MicroPy Pyboard Stub module: wipy to file: /stubs/micropython-v1_19_1-rp2/wipy.py Skip module: wipy Module not found.
MicroPy Pyboard Stub module: writer to file: /stubs/micropython-v1_19_1-rp2/writer.py Skip module: writer Module not found.
MicroPy Pyboard Stub module: xpt2046 to file: /stubs/micropython-v1_19_1-rp2/xpt2046.py Skip module: xpt2046 Module not found.
MicroPy Pyboard Stub module: ymodem to file: /stubs/micropython-v1_19_1-rp2/ymodem.py Skip module: ymodem Module not found.
MicroPy Pyboard Stub module: zephyr to file: /stubs/micropython-v1_19_1-rp2/zephyr.py Skip module: zephyr Module not found.
MicroPy Pyboard Stub module: zlib to file: /stubs/micropython-v1_19_1-rp2/zlib.py mem:116464
MicroPy Pyboard Created stubs for 48 modules on board micropython-v1.19.1-rp2
MicroPy Pyboard Path: /stubs/micropython-v1_19_1-rp2
MicroPy Pyboard Failed to create the report.
MicroPy ✔ Done!
MicroPy Copying stubs...
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/_boot_fat.py: 100%|██████████████| [172/172 @ 279B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/_onewire.py: 100%|███████████████| [309/309 @ 302B/s]
I grabbed a copy of the files that had been copied to the temp directory at the point the script fails, interestingly the last few are 0 bytes, these are also the files that display "[0.00/0.00 @ ?B/s]" in the micropy output. The file names also stop before the end of the list above - there's nothing after onewire
, even though there are other packages listed in the stub creation output from micropy (e.g. uzlib
, version
, websocket
etc. Is there any chance it's running out of RAM/Flash on the board?
-rw-r--r-- 1 rebeam staff 344B Mar 23 10:33 _boot_fat.py
-rw-r--r-- 1 rebeam staff 618B Mar 23 10:33 _onewire.py
-rw-r--r-- 1 rebeam staff 1.9K Mar 23 10:33 _rp2.py
-rw-r--r-- 1 rebeam staff 849B Mar 23 10:33 _thread.py
-rw-r--r-- 1 rebeam staff 732B Mar 23 10:33 _uasyncio.py
-rw-r--r-- 1 rebeam staff 542B Mar 23 10:33 adcfft.py
-rw-r--r-- 1 rebeam staff 536B Mar 23 10:33 array.py
-rw-r--r-- 1 rebeam staff 579B Mar 23 10:33 binascii.py
-rw-r--r-- 1 rebeam staff 1.6K Mar 23 10:33 breakout_as7262.py
-rw-r--r-- 1 rebeam staff 1.0K Mar 23 10:33 breakout_bh1745.py
-rw-r--r-- 1 rebeam staff 1.2K Mar 23 10:33 breakout_bme280.py
-rw-r--r-- 1 rebeam staff 1.4K Mar 23 10:33 breakout_bme68x.py
-rw-r--r-- 1 rebeam staff 1.3K Mar 23 10:34 breakout_bmp280.py
-rw-r--r-- 1 rebeam staff 933B Mar 23 10:34 breakout_dotmatrix.py
-rw-r--r-- 1 rebeam staff 1.1K Mar 23 10:34 breakout_encoder.py
-rw-r--r-- 1 rebeam staff 769B Mar 23 10:34 breakout_icp10125.py
-rw-r--r-- 1 rebeam staff 2.0K Mar 23 10:34 breakout_ioexpander.py
-rw-r--r-- 1 rebeam staff 1.5K Mar 23 10:34 breakout_ltr559.py
-rw-r--r-- 1 rebeam staff 678B Mar 23 10:34 breakout_matrix11x7.py
-rw-r--r-- 1 rebeam staff 1.5K Mar 23 10:34 breakout_mics6814.py
-rw-r--r-- 1 rebeam staff 2.5K Mar 23 10:34 breakout_msa301.py
-rw-r--r-- 1 rebeam staff 1.1K Mar 23 10:34 breakout_paa5100.py
-rw-r--r-- 1 rebeam staff 1.1K Mar 23 10:34 breakout_pmw3901.py
-rw-r--r-- 1 rebeam staff 1.1K Mar 23 10:34 breakout_potentiometer.py
-rw-r--r-- 1 rebeam staff 681B Mar 23 10:34 breakout_rgbmatrix5x5.py
-rw-r--r-- 1 rebeam staff 4.2K Mar 23 10:34 breakout_rtc.py
-rw-r--r-- 1 rebeam staff 808B Mar 23 10:34 breakout_scd41.py
-rw-r--r-- 1 rebeam staff 1.1K Mar 23 10:34 breakout_sgp30.py
-rw-r--r-- 1 rebeam staff 1.1K Mar 23 10:34 breakout_trackball.py
-rw-r--r-- 1 rebeam staff 1.7K Mar 23 10:34 breakout_vl53l5cx.py
-rw-r--r-- 1 rebeam staff 781B Mar 23 10:34 cmath.py
-rw-r--r-- 1 rebeam staff 1.3K Mar 23 10:34 collections.py
-rw-r--r-- 1 rebeam staff 540B Mar 23 10:35 cryptolib.py
-rw-r--r-- 1 rebeam staff 1.0K Mar 23 10:35 dht.py
-rw-r--r-- 1 rebeam staff 781B Mar 23 10:35 ds18x20.py
-rw-r--r-- 1 rebeam staff 1.3K Mar 23 10:35 encoder.py
-rw-r--r-- 1 rebeam staff 939B Mar 23 10:35 errno.py
-rw-r--r-- 1 rebeam staff 1.3K Mar 23 10:35 framebuf.py
-rw-r--r-- 1 rebeam staff 665B Mar 23 10:35 gc.py
-rw-r--r-- 1 rebeam staff 4.1K Mar 23 10:35 gfx_pack.py
-rw-r--r-- 1 rebeam staff 734B Mar 23 10:35 hashlib.py
-rw-r--r-- 1 rebeam staff 479B Mar 23 10:35 heapq.py
-rw-r--r-- 1 rebeam staff 1.2K Mar 23 10:35 hub75.py
-rw-r--r-- 1 rebeam staff 5.0K Mar 23 10:35 interstate75.py
-rw-r--r-- 1 rebeam staff 1.6K Mar 23 10:35 io.py
-rw-r--r-- 1 rebeam staff 859B Mar 23 10:35 jpegdec.py
-rw-r--r-- 1 rebeam staff 513B Mar 23 10:35 json.py
-rw-r--r-- 1 rebeam staff 1.9K Mar 23 10:35 lwip.py
-rw-r--r-- 1 rebeam staff 10K Mar 23 10:35 machine.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:35 math.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:36 micropython.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:36 mip.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:36 motor.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:36 neopixel.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:36 network.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:36 ntptime.py
-rw-r--r-- 1 rebeam staff 0B Mar 23 10:36 onewire.py
I've now tried again with a Pico Lipo with 16MB of flash, and that completes successfully - the Pico W only has 2MB of flash, and I believe some additional libraries. Of course it might be something other than the flash size causing the difference...
@trepidacious Thanks for the updates.
Your suspicion on the flash size is certainly plausible.
I've just pushed out ~https://github.com/BradenM/micropy-cli/releases/tag/v4.2.0-beta.1~ https://github.com/BradenM/micropy-cli/releases/tag/v4.2.0-beta.2 (had a minor issue with v4.2.0.beta.1 release...) which exposes additional flags to the create
command.
Namely, this includes --variant
, which allows you to use a couple of different versions of micropython-stubber through the use of codemods (see: https://github.com/Josverl/micropython-stubber/pull/298 for more details).
Although compiling w/ mpy-cross would be ideal here -- I have not yet implemented it. Meanwhile, I'm curious to see what results you achieve toying around with the variant option and the two provide backends. If you find the time, please let me know what you find.
Thanks!
@BradenM
Hi, thanks for the update - I've repeated the test on a Pico W with each of the options for --variant
('base', 'mem', 'db', 'lvgl'), and unfortunately they all give the same error (looks to be the same in each case, and very similar to the error form the previous version):
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/hub75.py: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| [633/633 @ 442B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/interstate75.py: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| [2.52k/2.52k @ 547B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/io.py: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| [838/838 @ 456B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/jpegdec.py: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| [429/429 @ 420B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/json.py: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| [256/256 @ 250B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/lwip.py: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| [985/985 @ 538B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/machine.py: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| [5.00k/5.00k @ 609B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/math.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/micropython.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/mip.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/motor.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/neopixel.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/network.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/ntptime.py: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Reading /stubs/micropython-v1_19_1-rp2/onewire.py: | | [0.00/0.00 @ ?B/s]
MicroPy Copied Stubs: stubs
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /opt/homebrew/lib/python3.10/site-packages/micropy/app/stubs.py:140 in stubs_create │
│ │
│ 137 │ │ out_dir = Path(tmpdir) │
│ 138 │ │ stub_path = next(out_dir.iterdir()) │
│ 139 │ │ log.info(f"Copied Stubs: $[{stub_path.name}]") │
│ ❱ 140 │ │ stub_path = mp.stubs.from_stubber(stub_path, out_dir) │
│ 141 │ │ stub = mp.stubs.add(str(stub_path)) │
│ 142 │ pyb.disconnect() │
│ 143 │ log.success(f"Added {stub.name} to stubs!") │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ _get_desc = <function stubs_create.<locals>._get_desc at 0x1158d84c0> │ │
│ │ backend = <CreateBackend.upydevice: 'upydevice'> │ │
│ │ create_stubs = <_io.StringIO object at 0x11595c550> │ │
│ │ ctx = <click.core.Context object at 0x11592ceb0> │ │
│ │ exclude = [] │ │
│ │ log = <micropy.logger.ServiceLog object at 0x11585b760> │ │
│ │ message_handler = MessageHandlers( │ │
│ │ │ on_message=<function stubs_create.<locals>.<lambda> at 0x1158d85e0>, │ │
│ │ ) │ │
│ │ module = [] │ │
│ │ mp = <micropy.main.MicroPy object at 0x11585afb0> │ │
│ │ out_dir = PosixPath('/var/folders/vz/pbzdmz6x6xb87qnyslnwptwc0000gn/T/tmpt6pg1ojr') │ │
│ │ port = '/dev/cu.usbmodem21201' │ │
│ │ pyb = <micropy.pyd.pydevice.PyDevice object at 0x11592d540> │ │
│ │ pyb_log = <micropy.logger.ServiceLog object at 0x11592d4e0> │ │
│ │ stub_path = PosixPath('/var/folders/vz/pbzdmz6x6xb87qnyslnwptwc0000gn/T/tmpt6pg1ojr/s… │ │
│ │ tmpdir = '/var/folders/vz/pbzdmz6x6xb87qnyslnwptwc0000gn/T/tmpt6pg1ojr' │ │
│ │ variant = <CreateStubsVariant.BASE: 'base'> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/homebrew/lib/python3.10/site-packages/micropy/stubs/stubs.py:355 in from_stubber │
│ │
│ 352 │ │ """ │
│ 353 │ │ _path = Path(path).resolve() │
│ 354 │ │ dest = Path(dest).resolve() │
│ ❱ 355 │ │ mod_file = next(_path.rglob("modules.json")) │
│ 356 │ │ path = mod_file.parent │
│ 357 │ │ mod_data = json.load(mod_file.open()) │
│ 358 │ │ dev_fware = mod_data["firmware"] │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ _path = PosixPath('/private/var/folders/vz/pbzdmz6x6xb87qnyslnwptwc0000gn/T/tmpt6pg1ojr/stu… │ │
│ │ dest = PosixPath('/private/var/folders/vz/pbzdmz6x6xb87qnyslnwptwc0000gn/T/tmpt6pg1ojr') │ │
│ │ path = PosixPath('/var/folders/vz/pbzdmz6x6xb87qnyslnwptwc0000gn/T/tmpt6pg1ojr/stubs') │ │
│ │ self = <micropy.stubs.stubs.StubManager object at 0x1159fc610> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
StopIteration
@trepidacious Thank you for the update!
Looking at this now, it seems clear what is most likely occurring.
The pydevice backend has logic built into it that attempts to circumvent OOM errors by automatically rebooting the device and reducing the buffer size used for reading from the device.
However, this logic works with the expectation that reads (and/or pydevice) throw exceptions / errors during the read. What seems to be happening is it's just reading nothing.
So, while it remains true that compiling beforehand via mpy-cross would likely prevent (not solve) this issue, I will revisit this logic and implement improved checks to ensure the integrity of the files. This will also be needed for https://github.com/BradenM/micropy-cli/issues/312#issuecomment-1456769622 and related issues (and its probaby something I should have done from the get go... :grimacing: )
Thank you again @trepidacious for your time in helping me iron these issues out. Much appreciated.
@trepidacious Hi, finally found the time to get another release out.
Mpy cross compilation and file integrity verification are in, so this should be the one 🤞
Going to go ahead and close this out. Please reach out to me if any issues arise and it needs to be opened, thanks!
I've been running into this for a few hours now. Here's my logs.
▶ micropy stubs create /dev/cu.usbmodem1301
MicroPy Connecting to Pyboard @ /dev/cu.usbmodem1301
MicroPy ✔ Connected!
2023-06-17 13:39:09.577 | DEBUG | stubber.minify:minify_script:253 - Original length : 26905
2023-06-17 13:39:09.577 | INFO | stubber.minify:minify_script:254 - Minified length : 11631
2023-06-17 13:39:09.577 | INFO | stubber.minify:minify_script:255 - Reduced by : 15274
2023-06-17 13:39:10.854 | ERROR | stubber.minify:cross_compile:330 - mpy-cross failed to compile:
MicroPy Executing stubber on pyboard...
MicroPy Pyboard Writing /createstubs.mpy: | | [0.00/0.00 @ ?B/s]
MicroPy Pyboard Traceback (most recent call last):
MicroPy Pyboard File "<stdin>", line 1, in <module>
MicroPy Pyboard ValueError: incompatible .mpy file
MicroPy ✔ Done!
MicroPy Copying stubs...
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/marcusstenbeck/.pyenv/versions/3.9.17/lib/python3.9/site-packages/micropy/app/stubs.py:18 │
│ 7 in stubs_create │
│ │
│ 184 │ log.success("Done!") │
│ 185 │ log.info("Copying stubs...") │
│ 186 │ with tempfile.TemporaryDirectory() as tmpdir: │
│ ❱ 187 │ │ pyb.copy_from( │
│ 188 │ │ │ DevicePath("/stubs"), │
│ 189 │ │ │ tmpdir, │
│ 190 │ │ │ verify_integrity=True, │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ _get_desc = <function stubs_create.<locals>._get_desc at 0x10ac0b9d0> │ │
│ │ backend = <CreateBackend.upydevice: 'upydevice'> │ │
│ │ compile = True │ │
│ │ create_stubs = <_io.BytesIO object at 0x10afeff40> │ │
│ │ ctx = <click.core.Context object at 0x10ac98a00> │ │
│ │ dev_path = 'createstubs.mpy' │ │
│ │ exclude = [] │ │
│ │ exclude_defaults = True │ │
│ │ log = <micropy.logger.ServiceLog object at 0x10ac6c9a0> │ │
│ │ message_handler = MessageHandlers( │ │
│ │ │ on_message=<function stubs_create.<locals>.<lambda> at 0x10ac0baf0>, │ │
│ │ ) │ │
│ │ module = [] │ │
│ │ module_defaults = True │ │
│ │ mp = <micropy.main.MicroPy object at 0x10ac6cb20> │ │
│ │ port = '/dev/cu.usbmodem1301' │ │
│ │ pyb = <micropy.pyd.pydevice.PyDevice object at 0x10ac984f0> │ │
│ │ pyb_log = <micropy.logger.ServiceLog object at 0x10ac98b80> │ │
│ │ tmpdir = '/var/folders/z0/s8bx3n216sl2s1qqg622rddc0000gn/T/tmpijehse71' │ │
│ │ variant = <CreateStubsVariant.BASE: 'base'> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /Users/marcusstenbeck/.pyenv/versions/3.9.17/lib/python3.9/site-packages/micropy/pyd/pydevice.py │
│ :50 in copy_from │
│ │
│ 47 │ │ src_path = Path(str(source_path)) │
│ 48 │ │ # 'is_dir/file' only works on existing paths. │
│ 49 │ │ if not src_path.suffix: │
│ ❱ 50 │ │ │ return self.pydevice.copy_dir( │
│ 51 │ │ │ │ DevicePath(source_path), │
│ 52 │ │ │ │ target_path, │
│ 53 │ │ │ │ consumer=self.consumer, │
│ │
│ ╭────────────────────────────────────── locals ──────────────────────────────────────╮ │
│ │ exclude_integrity = {'usys.py', 'sys.py'} │ │
│ │ self = <micropy.pyd.pydevice.PyDevice object at 0x10ac984f0> │ │
│ │ source_path = '/stubs' │ │
│ │ src_path = PosixPath('/stubs') │ │
│ │ target_path = '/var/folders/z0/s8bx3n216sl2s1qqg622rddc0000gn/T/tmpijehse71' │ │
│ │ verify_integrity = True │ │
│ ╰────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /Users/marcusstenbeck/.pyenv/versions/3.9.17/lib/python3.9/site-packages/micropy/pyd/backend_upy │
│ device.py:157 in copy_dir │
│ │
│ 154 │ │ target_path = Path(str(target_path)) # type: ignore │
│ 155 │ │ source_path = self.resolve_path(source_path) │
│ 156 │ │ exclude_integrity = exclude_integrity or set() │
│ ❱ 157 │ │ for file_path in self.iter_files(source_path): │
│ 158 │ │ │ rel_path = PurePosixPath(file_path).relative_to( │
│ 159 │ │ │ │ list(PurePosixPath(file_path).parents)[-1] │
│ 160 │ │ │ ) │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ exclude_integrity = {'usys.py', 'sys.py'} │ │
│ │ kwargs = { │ │
│ │ │ 'consumer': <micropy.pyd.consumers.ConsumerDelegate object at │ │
│ │ 0x10ac98ca0>, │ │
│ │ │ 'verify_integrity': True │ │
│ │ } │ │
│ │ self = <micropy.pyd.backend_upydevice.UPyDeviceBackend object at 0x10ac98970> │ │
│ │ source_path = '/stubs' │ │
│ │ target_path = PosixPath('/var/folders/z0/s8bx3n216sl2s1qqg622rddc0000gn/T/tmpijehse71… │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /Users/marcusstenbeck/.pyenv/versions/3.9.17/lib/python3.9/site-packages/micropy/pyd/backend_upy │
│ device.py:140 in iter_files │
│ │
│ 137 │ │ if not results: │
│ 138 │ │ │ return │
│ 139 │ │ for file_result in results: │
│ ❱ 140 │ │ │ name, type_, _, _ = file_result │
│ 141 │ │ │ abs_path = PurePosixPath(path) / name │
│ 142 │ │ │ if type_ == stat.S_IFDIR: │
│ 143 │ │ │ │ yield from self.iter_files(abs_path) │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ file_result = 'T' │ │
│ │ path = '/stubs' │ │
│ │ results = 'Traceback (most recent call last):\n File "<stdin>", line 1, in │ │
│ │ <module>\nOSError'+19 │ │
│ │ self = <micropy.pyd.backend_upydevice.UPyDeviceBackend object at 0x10ac98970> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: not enough values to unpack (expected 4, got 1)
Looks like mpy cross compile is plain failing, and it's not possible to diable the cross compiling via command lien options either.
Describe the bug
micropy fails to generate stubs from a PicoW board running Pimoroni Micropython, when no other stubs exist.
Expected Behavior
Micropy generates stubs successfully.
Current Behavior
Micropy produces a
TypeError
and doesn't generate the stubs.Steps to Reproduce
rm -rf ~/.micropy
micropy stubs create /dev/cu.usbmodem11201
Possible Solution
For me, adding
str
aroundname
in the line"/".join([package.repo_name, name])
inresolve_package
function ofrepo.py
fixed the issue.Once the stubs have generated once with this change, the unmodified code works fine - seems to be something about having no stubs, or maybe not having an existing version of the same-named stubs?
Logs
The last section of output from
micropy stubs create /dev/cu.usbmodem11201
- everything before this looks normal:micropy.log
Context (Environment)