JetBrains / intellij-micropython

Plugin for MicroPython devices in PyCharm and IntelliJ
https://plugins.jetbrains.com/plugin/9777-micropython
Apache License 2.0
506 stars 106 forks source link

Only main.py flashes on Raspberry Pi Pico #225

Open jakobbp opened 1 year ago

jakobbp commented 1 year ago

Only main.py flashes successfully on Raspberry Pi Pico. Files with any other name do not, despite Run configuration and output looking identical. What gives?

Using PyCharm CE 2022.3.1 and MicroPython 1.4.0

gabemorris12 commented 1 year ago

I'm pretty sure that's just how it's designed to work. What's the issue with just calling the file main.py?

jakobbp commented 1 year ago

That's how what is designed to work? The MicroPython plugin? Because, if it is (of which I am seriously doubtful), that is not a design, but just stupidity.

Would you like to name all your programs main? If you would, why stop there and not name all your functions f0, f1, f2, ... and your variables a, b, c, ... too?

gabemorris12 commented 1 year ago

As far I can tell, the plugin is designed to only flash scripts named main.py. I agree that the naming of files is important for readability and organization. I can see this being an issue if you have a project where you want to test multiple scripts.

My work around for this is to flash a directory instead of a single file. There are more details in the readme, but you can name the directory whatever you'd like according to its application, but the main.py script has to be inside the directory.

jakobbp commented 1 year ago

I just can not believe, that this behaviour would be intentional. Screenshots in the readme have files like microbit_example.py in them and despite your sensible "next best thing" workaround, this issue still completely defeats many of PyCharm's basic features, like tab browsing (every open file tab would have the same title) and shift+shift searching...

ukBaz commented 1 year ago

micropython has the requirement that there are two files on device. To quote from:

https://www.digikey.co.uk/en/maker/projects/micropython-basics-load-files-run-code/fb1fcedaf11e4547943abfdd8ad825ce

Boot Scripts

There are two important files that MicroPython looks for in the root of its filesystem. These files contain MicroPython code that will be executed whenever the board is powered up or reset (i.e. it 'boots'). These files are:

  1. /boot.py - This file is run first on power up/reset and should contain low-level code that sets up the board to finish booting. You typically don't need to modify boot.py unless you're customizing or modifying MicroPython itself. However it's interesting to look at the contents of the file to see what happens when the board boots up. Remember you can use the ampy get command to read this and any other file!
  2. /main.py - If this file exists it's run after boot.py and should contain any main script that you want to run when the board is powered up or reset.

The main.py script is what you can use to have your own code run whenever a MicroPython board powers up. Just like how an Arduino sketch runs whenever the Arduino board has power, writing a main.py to a MicroPython board will run that code whenever the MicroPython board has power.

There is also similar in the official docs at: https://docs.micropython.org/en/latest/pyboard/tutorial/script.html#opening-the-pyboard-usb-drive

So the entry point for micropython is a file called main.py. However that file could be a call to a file of a different name.

thumdugger commented 1 year ago

TL;DR

Just run the code manually and don't use directories (see at the bottom ... example is for a Linux box but can be adapted to your OS and environment ... note this will fail without some additional work that is briefly explained at the bottom of this post): python microupload.py -v /dev/ttyACM0 my_flash_file.py

main.py isn't the issue

@ukBaz There is no requirement for ONLY two files on the device just that if either, or both, of these files exist they will be acted on by the system in specific ways.

The first article cited makes extensive use of files NOT called main.py. The whole workflow presented there is based on writing code NOT named main.py so you can test code out until you are ready to make it the code that gets run whenever the device is reset.

Even the quoted block, item 2, starts the description with "If this file exists"; it doesn't ever need to exist and usually wouldn't until you are ready for the device to run code automatically after booting up.

In general you can have whatever files and directories you want and can run any of the code from any of those directories whenever you want. This is what Thonny does with your code no matter what it is named and it will happily move any validly name file over to the device and execute it.

The last paragraph in that first cite sums it up this way:

Putting all the pieces of this guide together you can see a simple workflow for MicroPython that's similar to Arduino & Arduino sketches:

  • Write Python code on your computer using your favorite text editor. Structure the code so it puts setup code at the top and loop code inside a main loop.
  • Use the ampy run command with the --no-output option to run the script on the MicroPython board.
  • Edit and run the script as much as you need for it to work the way you expect.
  • When you want the code to automatically run on boot use the ampy put command to save the script as a /main.py file on the board.

The third bullet point is the "run whatever file you want that is named whatever you want". The fourth bullet point is about how to make whatever code you want running when the machine is turned on actually start after booting up: name it main.py.

However, all of that said, it is not clear to me, at the moment, that the plugin restricts files to only be named main.py. I don't see any code anywhere doing that.

Some actual issues

However, the code does do something else very strange that it shouldn't be doing.

Currently, on my machine, it automatically determines the absolute path to the directory containing the file you are flashing from the path set in Flash configuration. So if the file you are trying to flash is at, /home/me/my_code/flash_me.py it will set the --chdir option of the actual update script to, /home/me/my_code ... which is fine ... unnecessary but fine.

But, it then also passes that very same full path in as the path to write to the device. So instead of flashing, flash_me.py, to the root directory of the device it instead tries to create the, /home/me/my_code, directories on the device and then write the file into the that directory chain.

On my system this usually just fails outright with an OSErr: 17 error. This appears to happen because the ampy.Files code this plugin ultimately relies on expects to see, OSError: [Errno 17] EEXIST, in order to successfully ignore the error but fails to because it received OSErr: 17, instead.

Sometimes it'll just hang although I think that hanging is actually the same OSErr issue just mentioned but it sometimes seems to get unstable in communicating with the device so weird things happen.

I know it is going to fail because the flash command runs as (for example):

/home/dad/share/coding/raspberrypi/pico/first_try/venv/bin/python /home/dad/.local/share/JetBrains/Toolbo /apps/PyCharm-P/ch-0/223.8617.48.plugins/intellij-micropython/scripts/microupload.py -C /home/dad/share/coding/raspberrypi/pico/first_try -v /dev/ttyACM0 /home/dad/share/coding/raspberrypi/pico/first_try/pico_lcd_114.py
Connecting to /dev/ttyACM0
Uploading files: 0% (0/1)
/home/dad/share/coding/raspberrypi/pico/first_try/pico_lcd_114.py -> ../../../../../../home/dad/share/coding/raspberrypi/pico/first_try/pico_lcd_114.py

Note the huge string of parent directories it prepended to flash file's path? That whole last line should probably be:

/home/dad/share/coding/raspberrypi/pico/first_try/pico_lcd_114.py -> pico_lcd_114.py

which, if I run these things like this manually it works fine.

I have not checked deep into it but the file system walker appears to be prepending entirely unnecessary parent paths to the entirely unnecessary full flash file path. I suspect the code tries to do a mkdir .. sort of call, or some variation on creating ./, which all results in EEXIST error because of course it already exists. This would all probably work fine if it ignored the EEXIST error but it can't because the error message is different than the one it is looking for.

All of which results in it usually not doing what you want it to do.

The plugin appears to behave this way because it passes the full file path in the variable, path, instead of rootDir.name in the getMicroUploadCommand in the plugin file MicroUpdate.kt.

The upload script appears to behave this way (no directories except maybe the first time you flash it) because the something changed that the ampy.files code hasn't been updated to deal with.

The Work Around

The workaround is to not use the Flash Run configuration since it doesn't work.

Instead, just run upload script manually since all the plugin is doing here is running it for you (incorrectly).

The quickest way to accomplish this is to just flash the file as normal and let it fail. Then copy the command out the Run history and remove all of the directory stuff preceding your file name:

Turn this failure:

> /home/dad/share/coding/raspberrypi/pico/first_try/venv/bin/python /home/dad/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/223.8617.48.plugins/intellij-micropython/scripts/microupload.py -C /home/dad/share/coding/raspberrypi/pico/first_try -v /dev/ttyACM0 testing/picoup.py

into this success:

> python /home/dad/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/223.8617.48.plugins/intellij-micropython/scripts/microupload.py -v /dev/ttyACM0 picoup.py

Note your failures will probably look different than mine so don't expect my example to work. This important part here is that you need to run the microupload.py python script, yourself, instead of through the Flash Run configuration. In this example I'm just using the one that came with the plugin but you could just as easily copy, or link, it to you coding directory and run it from there:

> ln -s /home/dad/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/223.8617.48.plugins/intellij-micropython/scripts/microupload.py microupload.py
> python microupload.py -v /dev/ttyACM0 picoup.py

You can, of course, adjust the upload script to run automagically on your box (#! /usr/bin/env python kind of stuff) and put it in your path (~/.local/bin, for example) and then that whole last line could just reduce to: `microupload.py -v YOUR_SERIAL_PORT_DEVICE YOUR_FILE_TO_FLASH

This will not, however, solve the sub-directory problem since if the directory exists the microupload.py will still fail because the ampy.files code it is dependent on is failing to correctly deal with the EEXIST error. That is a simple modification but that mod will be local to your box until it gets fixed (maybe, never?) and will be overwritten, when updated, if you change the code in place. Maybe best just to make your own local fork for the time being.

Heck, if you are truly ambitious then one might also just make a new Run configuration that still flashed at a click of a button ... but I haven't tried that yet so I don't know if that would work or not.

EDIT, RE: ampy.files issue

I did come across this pull request that was merged on 18 Mar 2021. This is not the same code that is in my virtual environment and I'm not currently sure why that is. However, the merged ampy.files code on that github would resolve the directories issues and just leave (maybe) the full-paths issues. I didn't knowingly install this older ampy.files code so either that is the most current code released or somehow the installation process of the plugin is pulling in older code that breaks microupload.py.

codiak6335 commented 1 year ago

With REPL closing every run there is no feedback loop when a module fails in the first millisecond. I was never able to get my test.py to initiate from the IDE... it copied over, but no start that I could see. Thonny is nice but as noted many times lacks features. Pycharm Pro has been my go-to IDE for years but the workflow for Micropython Pico is broken. Time to break down an start using VSC