Open askvictor opened 8 years ago
I would also like to see this.
For any boards that you can connect and get a REPL, one of the key Micropython developers built a tool that is useful and could be used as part of the interface or as a example on how to integrate Micropython into PlatformIO https://github.com/dhylands/rshell
I am also using the Pycom modules which has a fork of Micropthon. These boards include wifi, bluetooth 4.0, and soon cellular. All their newer boards are based on the ESP32 modules.
There is also an Atom package called pymakr that can provide access to both Serial and Web - REPL. It is supposed to syncronize (upload/download) files too - but this feature does not appear to be working in the latest release
Another useful source to consider in building Micropython suppport into https://github.com/nickzoic/mpy-utils
As I have been moving more toward Python lately, I miss the integration that Platformio/Atom had achieved. Is there anyone out there working on integrating micropython tools into Platformio?
Probably the most universal and straightforward way of doing this is by integrating, or doing something similar to rshell. This relies on the REPL which is a pretty universal method on connecting to Micropython devices. https://github.com/dhylands/rshell
Pycom does something similar with their Atom plugin, but they are focused solely on their products, so even if other devices work, they are not truly supported. https://atom.io/packages/pymakr
I do a bunch of Micropython on Wemos D1 units (ESP8266) so I really love the idea of support for this. Since it hasn't been linked elsewhere in the thread: MP ESP8266 Docs & Firmware Download
Is there a timeline for micropython on platformio?
Based on how I have used Micropython and VScode what would cover most use cases is
Then being able to open open a reply using one of the methods as well.
For now I write a small script that uses rsync and a file to list what needs to be copied over. I use the standard pyboard based implementations (USB) most of the time so it is mounted as a VCP and a mountable drive through the same connection. I have a pretty quick edit and deploy processes now.
Uploading lower level firmware is so uncommon for me that I don't think that needs to be initially supported.
@TravisJoe rshell be used to get the devices repl from remote network over the internet? I would like debug my micropython script using something like that.
@Niko-La rshell is designed for serial or virtual com port interactions with a micropython device. For connecting over a network you will want something more like this https://github.com/micropython/webrepl
I think Adafuits tool also can do over network https://learn.adafruit.com/micropython-basics-load-files-and-run-code/install-ampy
Micropython plz. I like Atom.
+1
Platform.io + Micropython. Pretty please!!
+1
+1
+1
+1 for vscode
Hey folks, could you please stop writing comments "+1"? Here are "likes" emoji for that. Comment will send notification to all subscribed users.
Jan Pobořil https://honza.poboril.cz/
+1 for vscode
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
Hi all,
Could you explain about classic workflow how you do programming using Mycropython? To be honest, I don't understand what should be done from our side.
I would be thankful if you explain what you expect from PlatformIO.
Thanks!
Dear Ivan!
My opinion, that the micropython (mp) support doesn't differ from the other supported platform framework . The plaformio downloads the mp's code, builds it with the mp's tool, flashes the firmware and upload the app specific code. The mp has a package handling tool called upip (after the Python pip). The plarformio IDE should support the mp REPL - it can be usable on the connected microcontroller flashed with mp. I think the debugging could be solved with mp also - but I haven't used the debugging yet, because the REPL very usable tool for testing codes directly on the microcontroller.
I'm only little experienced with mp, but the micropython developers can provide more detailed information and maybe can help in the implementation. The mp forum (https://forum.micropython.org/) and/or github repo (https://github.com/micropython/micropython) could be easiest way to contact with the mp gurus.
I hope I can help. Regards, Adrian
I understand MP is too different to support it in Platformio easilly.
My current workflow is to have Makefile for compiling all possible code as frozen modules (compiled and included in firmware) and after flashing copy custom main.py and config to filesystem.
MICROPYTHON_VERSION=v2.11.0.2
FLASHING_PORT=cu.usbserial
run-docker:
docker run -it --rm -v ${PWD}:/app registry.gitlab.com/janpoboril/micropython-docker-build/pycopy/esp8266:$(MICROPYTHON_VERSION) make build-firmware
build-firmware:
cp -R lib/* src/modules/* /micropython/ports/esp8266/modules
cd /micropython/ports/esp8266 && make
mv /micropython/ports/esp8266/build/firmware-combined.bin /app
install: firmware-combined.bin
esptool.py -p /dev/$(FLASHING_PORT) erase_flash
esptool.py -p /dev/$(FLASHING_PORT) -b 115200 write_flash 0 firmware-combined.bin
mpfshell -o $(FLASHING_PORT) -c lcd src; put main.py; put config.py; put networks.json; put webrepl_cfg.py
- 2019 v 10:27, cselti notifications@github.com:
Dear Ivan! My opinion, that the micropython (mp) support doesn't differ from the other supported platform. The plaformio downloads the mp's code, builds it with the mp's tool, flashes the firmware and upload the app specific code. The mp has a package handling tool called upip (after the Python pip). The plarformio IDE should support the mp REPL - it can be usable on the connected microcontroller flashed with mp. I think the debugging could be solved with mp also - but I haven't used the debugging yet, because the REPL very usable tool for testing codes directly on the microcontroller. I'm only little experienced with mp, but the micropython developers can provide more detailed information and maybe can help in the implementation. The mp forum (https://forum.micropython.org/ https://forum.micropython.org/) and/or github repo (https://github.com/micropython/micropython https://github.com/micropython/micropython) could be easiest way to contact with the mp gurus.
I hope I can help. Regards, Adrian
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/platformio/platformio-core/issues/728?email_source=notifications&email_token=AAEV6WAITAPAMNFC5YSWHQTQHDKVXA5CNFSM4CKSOZU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5Q6V7A#issuecomment-526510844, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEV6WGSTKBW3UPPP3C4EIDQHDKVXANCNFSM4CKSOZUQ.
@iBobik - As far as I know, the platformio works the following in an application project (RTOS, Arduino)
In case of micropython (because python's dynamic nature) the 3 usually necessary once and upload (which is simplified flashing) only the app specific micropython files. Except when the project has native code, that is interfaced to micropython, the 3. steps should repeat in every new project build+upload.
My point of view, that the difference is maybe between in your project's workflow and the platformio worklow - but I think your workflow can be divided to the 1,2,3,4 steps also too.
And in case of micropython (because the Python's REPL) the platformio IDE should support to connect the REPL through UART separately.
But the Pycom (creates ESP32 boards with micropython support) created an VS Code based IDE - the Pymakr (https://github.com/pycom/pymakr-vsc/). The Pymakr supports their boards only, it's doesn't works different ESP32 board, but it can be useful example how it's upload the applications files and connects the REPL on UART.
Hi all,
Could you explain about classic workflow how you do programming using Mycropython? To be honest, I don't understand what should be done from our side.
I would be thankful if you explain what you expect from PlatformIO.
Thanks!
Hello @ivankravets , I think what everyone is looking for is similar functionality to uPyCraft IDE. Flashing, uploading, REPL, running software, etc.
I use Micropython almost exclusively for commercial projects. Because of how Micropython is designed and used there are many ways that IDE could help, and there is a pretty clean path to building up functionality on PlatformIO to improve the use with Micropython.
One thing to make sure is clear, the devices has an interpreter on it, so compiling code is not required before hand. Adding cross compiler functionality can get into OS specific tools, Micropython version specific tools and could make things complex quickly. My suggestion is to focus first on the development cycle which usually includes a lot of writing code and synchronizing files to the device so they can be debugged.
The three main ways I can think of to synchronize files are
If you can get the files over to the device the next thing is having a reliable REPL connection. This again mainly one of three methods.
Once you have a quick way you can get files over to the device and reliable REPL for the embedded python interpreted you have covered a majority of the needs for most development.
I personally wrote a script that synchronizes files based on a manifest file in my project using rsync to the USB mass storage device. This runs in a terminal tab in vscode, then I have a serial terminal as well for the REPL.
Once you get past that, the next thing I would find useful is a cross platform DFU utility to program the device, which could use one or more of the main communication methods.
Again once you get into Micropython cross compiling and DFU firmware builds, this suddenly opens up a lot of possibilities. I build custom boards so my micropython repo is customized for those boards, then I need backwards compatibility for clients so I also have multiple micropython versions. The cross compiler would be nice, but is version specific (or at least can be) so you have to match your cross compiled code with your micropython implementation.
One bonus feature is having files that mimics or provides standard library definitions in it so that the linter does not have as much issues.
Thanks all for the useful comments! I also did some research. I found 2 popular micro Python frameworks (virtual machines):
Do you use other/own VMs?
1) All exiting docs and manuals force users to flash the target device with VM before. It is clear. 2) User personally manages *.py files and external modules/packages 3) User has a few options on how to "flash/deploy" code to a target device.
1) We keep the same PlatformIO's philosophy with a "framework" option. You will be able to control which VM and version to use
2) No need to bother with VMs because we know everything about your hardware board =...
and we can cross-compile true VM for the specific device directly on the user's host machine. And, yes, we can flash this device with Python VM
3) I see there is a tool called mpy-cross
. I think we can use it to compile raw Python code to bytecode .mpy, similar to .pyc on desktop machines.
4) We can do uploading of .mpy files. Yes, the .mpy files should be compiled with the same VM which resides on the target device.
I think we can start with the official MicroPython. It covers a lot of hardware https://github.com/micropython/micropython/tree/master/ports
I found a list of useful libraries/packages by @adafruit => https://github.com/adafruit/Adafruit_CircuitPython_Bundle/tree/master/libraries
It would be great to have them in our registry and do the same what we do with C/C++ libraries. So, extend PlatformIO Library Dependency Finder automatically deploy dependent libraries via cross-compilation depending on a VM type.
1) Create a new PlatformIO project:
pio init -b esp32dev --project-option="framework=python"
Why python
? There a lot of types: MicroPython, CircuitPython, Zerynth
2) Place project Python source code to src
folder
3) Build a project or deploy firmware
# first time it will build VM
pio run
# flash
pio run -t upload
And yes, no need to install any tools, compilers, dockers, etc. Just "upload" and see how board blinks.
Comments are welcome.
There is also fork of MP's original author @pfalcon: https://github.com/pfalcon/pycopy
My personal thought is that the generic development cycle is much more important for user experience than the firmware deployment cycle (which is much less frequent).
My assumption, and based on my experience, most people start with a development board that already has a version of firmware on it. Or they use a pre-built stable firmware for a specific board. The only reason I started building my own firmware instead of using the stable builds is because I make custom boards with custom board definitions.
An efficient "write code, deploy code, test code" cycle is what is missing for me and I have written scripts to aid in this. I would guess most users both experienced and not would benefit from the focus on this as well.
Cross compiling is rarely a requirement because of how the VM is setup. Only once my projects are tested do I cross compile or freeze my code. Having the files in .py format aid in debugging as tracebacks can literally say which line number is causing the issue.
My personal suggestion is keep it simple for now. I rarely need to build new firmware, maybe every couple of months, and I use Micropython nearly every day for work and projects.
What to do with 3rd party packages/libraries?
Do you use them? Are they compatible between all these micro Python implementations?
I have used some libraries, but few. And to make it more complex is that some libraries are port specific and some are not. I would say that external libraries (non-standard) are pretty use-case specific and most of the time I am writing my own or using the standard libraries.
I do agree that having a unified library and dependency system would help. Micropython has been working on and improving their "Machine" library to abstract more interfaces to help with making code and libraries cross port. But overall this is pretty rare for me. Once I get the library into my source folder (I cannot or do not use upip on my projects)
Micropython is close enough to python 3.5 some desktop libraries even work without editing.
For micropython's list of libraries look here as well https://github.com/micropython/micropython-lib
Here is some info that may be useful!
All CircuitPython libraries are compatible with all CircuitPython boards, the CircuitPython hardware API is identical across all devices. The CircuitPython hardware API is not compatible with MicroPython (which as mentioned above a different API, sometimes varies for each board)
All available libraries are submodule'd either here (in the main supported bundle) or in the community bundle.
Version management is handled by release tags on github and are published automatically to PyPI as well. The requirements.txt has the dependencies.
Can the "pio run" be extended with a new "sync" target to syncing only the project's python files? This target can accept parameters for different type of sync (USB, serial, network).
I created a VS Code keyboard shortcut (Ctrl+Alt+U) that uploads the current file to my network-connected Micropython device. Maybe something in here will help with this effort:
(Cited file locations are for Windows, obvs)
The file webrepl_cli.py
from https://github.com/micropython/webrepl somewhere on your computer. That location specified in the VS Code settings:
settings.json located in %UserProfile%\Appdata\Roaming\Code\User\ (other platforms)
{
a bunch of stuff that was already there,
"webrepl_cli": "C:/code/esp/webrepl/webrepl_cli.py"
}
+ Replace the above path with wherever you put the file.
The device (ESP8266 or ESP32 in my case but can be any Micropython device) is connected to the network and running Webrepl:
boot.py located in root directory on the Micropython device
import webrepl
webrepl.start()
tasks.json located in ProjectRoot\.vscode\
{
"version": "2.0.0",
"tasks": [
{
"label": "upload_webrepl",
"type": "shell",
"command": "${config:python.pythonPath} ${config:webrepl_cli} -p password ${fileBasename} ip-or-name.of.micropython.device:${fileBasename}"
}
]
}
+ Replace the above IP address or name with the network address for your Micropython device.
+ e.g. 192.168.0.123
+ e.g. light-switch.cable.rcn.com
keybindings.json located in %UserProfile%\Appdata\Roaming\Code\User
// Place your key bindings in this file to override the defaultsauto[]
[
a bunch of stuff that was already there,
{
"key": "ctrl+alt+u",
"command": "workbench.action.tasks.runTask",
"args": "upload_webrepl"
}
]
would be nice to Circuitpython to this issue as well.
Is it possible to get usb serial monitor working with circuitpython? Circuitpython I believe is simple to deploy code to as the hardware just show up as a mass storage and the script are just written to it.
+1, can't wait for micropython support with platform IO.
@ivankravets I would like to accelerate the implementation by contributing. Is there any branch I can start working on? (like, an internal development branch that's unfinished) Just to be clear, if it happens it will be my personal project, (it will be slow)I can't do it in my work hours.
Also, I found adding stubs for hype tinting helps a lot. Here's what I'm using right now: micropy-cli, micropy-stubs, Pico-Stub. I guess the manager (micropy-cli) could be implemented into platformIO (either running in the background or make a similar package), so that selecting boards or platforms will automatically generate corresponding stubs?
It would be awesome to get micropython support for platformio :)
@ivankravets, @SubaruArai While I am not new to developing stuff, I am new to the maker world, and to me it seems like a great opportunity gone to waste if not at least some starting point for MicroPython support development was established to get things going.
A growing number of projects and supplier example code are now MicroPython only, which forces newbies like me away to Arduino IDE, and without newbies, all projects eventually loses traction. Where could one start helping out?
@nicklasb Not sure what you mean "forces newbies like me away to Arduino IDE". I develop my MicroPython projects using VS Code, which is much nicer than Adruino IDE.
@stefan-sherwood Just to be clear, I'm not just considering the language, but the integration with the boards and so on, the things platform IO does for C++. What is your setup?
See my previous comment: https://github.com/platformio/platformio-core/issues/728#issuecomment-537206555
What else specifically are you looking for?
Maybe I am wrong, but doesn’t PlatformIO offer a lot more functionality than that? Or have I misunderstood something completely?
I think I have a good perspective of several sides of this argument. I use Micropython for testing as well as some professional products that are in production.
For the sake of just getting started with Micropython PlatformIO might be too big, and VS Code with the serial terminal turned on is extremely simple and easy to use. The annoying part I get around with a script I wrote which which syncs files between my device and my active directory (which is also versioned with Git). That handles almost all of my development on a board that is already defined. If the board is not defined you have to go build firmware for it.
Now where PlatformIO would be great is the build system and automating that. Being able to quickly build a new "Board" definition folder would be great. Minimizing the complexity and learning curve there with ideally visual tools would be helpful even to me. Then as a product matures it is much better to compile your python code into the firmware and if PlatformIO could help manage both the application and build process to put it all together with a custom board definition would be amazing.
I have done something similar using Docker where in my Git project I have the board file defined, and a build config file which includes the names and paths to all the files I want compiled into my firmware. VS Code lets you run Docker as a test space and I can run my scripts with a Docker image that includes the entire build system already setup. It will move the board definition and application files over and compile the MBOOT and Firmware DFU files and save it into the project folder which is retained even after Docker is closed.
This one is not setup to run the latest Micropython version but it could easily be updated for that. https://hub.docker.com/repository/docker/travistconsulting/travist_docker_micropython
Here is a rundown of what I would expect Platformio to do for a Micropython project:
esptool.py
and rshell
$ pip install esptool rshell
pyboard.py
The pyboard.py
utility comes preinstalled with this repository.
Here is the official pyboard.py
documentation.
From within the cloned repository folder, copy the firmware scripts main.py
and lib.py
to the device flash drive, called :
$ pyboard.py --device /dev/ttyUSB2 -f cp main.py lib.py :
The correct device port was derived from the dmesg
device messages on Linux.
Library files can better be grouped in a /flash/lib
folder. If so, a boot.py
file needs to be added to the root /flash
folder of the device, with the following contents:
import sys
sys.path[1] = '/flash/lib'
rshell
to Access the Micropython REPLA device running Micropython has a shell environment, called the read–eval–print loop (REPL).
It can be accessed by running the rshell
tool.
$ rshell -p /dev/ttyUSB2 -b 115200
repl
repl
followed by a Return, hit Ctrl+D to soft reboot the device.main.py
script running on the device.rshell
.rshell
and to return to the command line.Is there an update on this? I am the principal maintainer for the Reflections project. It’s an open-source wrist watch with a round ESP32-based main board. See github and ReflectionsOS. I’m considering writing the project OS in MicroPython to make it easy to write, maintain, and teach. Next I need a MacOS compatible IDE to develop my MicroPython ReflectionsOS. Considering CircuitPython, Platform.io, Arduino IDE, Thonny IDE. It seems ironic to me that Platform.io is written in Python and doesn’t yet support MicroPython development. -Frank
@frankcohen I use Micropython and Circuitpython in production systems, and after the years since posting one of the first comments on this thread saying I want this also, I can also see how once you get a flow and IDE is not as critical as it feels initially. That said something that helps sync projects to the board and streamline that would be nice. But once you have a basic setup it works quite well in development. I like the idea of a manifest for what files are synced and having a user friendly REPL built in.
Mac and Linux you can use rsync for example to sync files to the flash. A basic python script can do this, and VSCode has an easy what to have a terminal in the bottom of your screen.
I have thought about making more utilities and make them more generic for different setups, but have not had the time.
Probably not the best answer, but there is at least options that could be used for the project and teaching.
Thanks Travis. I appreciate your thoughts and experience. Whatever I choose to do for Micropython ESP32 workflow I need it to be easy to set-up, so that I may teach the environment easily. -Frank
Frank Cohen, Founder @.*** http://www.votsh.com Phone: (408) 364-5508 USA
On Jan 15, 2023, at 10:26 AM, Travis Travelstead @.***> wrote:
@frankcohen https://github.com/frankcohen I use Micropython and Circuitpython in production systems, and after the years since posting one of the first comments on this thread saying I want this also, I can also see how once you get a flow and IDE is not as critical as it feels initially. That said something that helps sync projects to the board and streamline that would be nice. But once you have a basic setup it works quite well in development. I like the idea of a manifest for what files are synced and having a user friendly REPL built in.
Mac and Linux you can use rsync for example to sync files to the flash. A basic python script can do this, and VSCode has an easy what to have a terminal in the bottom of your screen.
I have thought about making more utilities and make them more generic for different setups, but have not had the time.
Probably not the best answer, but there is at least options that could be used for the project and teaching.
— Reply to this email directly, view it on GitHub https://github.com/platformio/platformio-core/issues/728#issuecomment-1383219040, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUEWMPM2F6XIV5YI4FGCLLWSQ6MFANCNFSM4CKSOZUQ. You are receiving this because you were mentioned.
"[…] after the years since posting one of the first comments on this thread saying I want this also, I can also see how once you get a flow and IDE is not as critical as it feels initially."
The above may be true for seasoned makers & hackers, but this is certainly not true for aspiring programmers like children and their educators in the educational system. For the latter groups, a smooth IDE workflow is essential for a good uptake and Micropython does an extremely poor job at it. By consequence, the uptake of Raspberry Pi and Arduino is several orders higher.
A programming language of high educational value like Python deserves much better…
For those use cases, circuitpython might be a better language. Very easy to set up the ide called 'mu' and very easy to flash, as most compatible boards appear to other devices as a drive. When you copy or save your updated python files, the board automatically reboots, and it starts running your new code. Also, they do native USB and HID.
Christian
On Mon, Jan 16, 2023, 8:24 AM Serge Y. Stroobandt @.***> wrote:
"[…] after the years since posting one of the first comments on this thread saying I want this also, I can also see how once you get a flow and IDE is not as critical as it feels initially."
The above may be true for seasoned makers & hackers, but this certainly not true for aspiring programmers like children and their educators in the educational system. For the latter groups, a smooth IDE workflow is essential for a good uptake and Micropython does an extremely poor job at it. By consequence, the uptake of Raspberry Pi and Arduino is several orders higher.
A programming language of high educational value like Python deserves much better…
— Reply to this email directly, view it on GitHub https://github.com/platformio/platformio-core/issues/728#issuecomment-1384142296, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEAGGGJ3PNZQSJ7MSFPBOK3WSVKYZANCNFSM4CKSOZUQ . You are receiving this because you commented.Message ID: @.***>
I just got started poking around with my new Pico W and while it wasn't too laborious to get a circuitpython-specific VSCode plugin running (which comes with built-in serial monitor), the one thing that I really missed as a career software developer who's gotten used to making sure all his projects are kicked up to github is that literally the only place the source code for a Pico board exists is on the board itself. And while I know I could probably git init
right on the mounted volume, I do worry about the 2mb flash limit. Also having a way to compile down to .mpy
as part of a build process would be kinda nice rather than having to manually install the entire toolchain.
Also keenly aware that setting up a full-blown dev and build environment with source control is maybe a little overkill if the primary audience is hobby/education. PIO could be an overengineered solution in search of a problem. Having said that, it would be nice to have the option of scaling up.
Add support for Micropython for compatible boards (e.g. esp8266 family)