esp-rs / embuild

Build support for embedded Rust: Cargo integration with other embedded build ecosystems & tools, like PlatformIO, CMake and kconfig.
Apache License 2.0
135 stars 38 forks source link

cargo-pio PlatformIO-first build with Arduino SDK #60

Closed DanielRoeven closed 2 years ago

DanielRoeven commented 2 years ago

I've gotten a cargo-pio PlatformIO-first project to build with the Arduino SDK for ESP32. It wasn't quite as straightforward as I initially thought, based on the boilerplate code for a new cargo-pio project.

I'll describe what I did first, and if anybody is willing to help me figure out how to support the Arduino SDK in cargo-pio properly, I'd be happy to make a PR for it, if so desired.

  1. Make a new project and set the frameworks flag to Arduino:
    cargo pio new <your-project-name> --board <your-board> --frameworks arduino
  2. If we build now (pio run), we'll get an error saying undefined reference to setup() and undefined reference to loop(). Surprising, at first, since if we look in lib.rs, there sure is an extern "C" fn setup() and an extern "C" fn arduino_loop() defined. (The latter has an export_name = loop pragma so as to call it "loop" for the linker so Arduino's SDK can find it). I could not get these Rust-defined extern "C" functions to work. And as I understand it, they really can't, either, since the Arduino SDK is written in C++, and you can't FFI between Rust and C++ like that... something to do with C++'s name mangling or ABI or something (this goes a little over my head). So my solution is to define setup() and loop() in C++:
  3. Rename dummy.c to dummy.cpp and include a stub for setup() and loop(). The error from step 2 disappears.
  4. A new error appears! KeyError : 'framework-arduino'. This originates from line 73 in platformio.cargo.py:
            env["ENV"]["CARGO_PIO_BUILD_PIO_FRAMEWORK_DIR"] = env.PioPlatform().get_package_dir("framework-" + env.GetProjectOption("framework")[0])
  5. What's going on here is that cargo is trying to add a framework by the name of framework-arduino, since it's a relatively straightforward concatenation it's doing. But this framework / directory doesn't exist! If we look in ~/.platformio/packages/ the directory is actually called framework-arduinoespressif32. And indeed, if we change line 73 in platformio.cargo.py to:
        env["ENV"]["CARGO_PIO_BUILD_PIO_FRAMEWORK_DIR"] = env.PioPlatform().get_package_dir("framework-arduinoespressif32")
  6. It all works!

Of course, our entry point is now in C++ and not in Rust, which is... not ideal since this whole ordeal was about developing for these boards with Rust, while using C(++) libs. But it's not such a big deal, since we can call back into Rust with FFI from our C++ setup() and loop() functions.

But it was a bit of a confusing journey. The presence of Arduino functions and the 'soft coded' way of linking PIO frameworks into the Cargo build really seemed to indicate that this was a supported use case. Am I missing something obvious? πŸ™‚

In the case that I'm not missing anything obvious, and using the Arduino SDK wasn't really a supported use case yet, I'd be happy to create a PR for this. But how would I go about it? It requires adding a C++ file (which we don't want in the esp-idf case), and hardcoding the Arduino for ESP32 framework name in the python script. Neither of which seem ideal.

Any thoughts?

jendakol commented 2 years ago

Hi @DanielRoeven, you're obviously more experienced with this kind of stuff - you saved me some hair with this post πŸ™‚ However, would you be so kind to leave somewhere a working code where the "call back into Rust with FFI from our C++ setup() and loop() functions works? πŸ™πŸ» Do you have any experience with using PlatformIO dependencies from the Rust code later?

As to your original question πŸ˜ƒ, obviously, I'm just a visitor here but I think that the Arduino framework path is supported, it just doesn't work πŸ™‚ (TBH the espidf framework didn't work too for me).

DanielRoeven commented 2 years ago

you're obviously more experienced with this kind of stuff

Hardly, I've just started playing with this. Hence my questions to the maintainers here πŸ™‚

call back into Rust with FFI from our C++ setup() and loop() functions

// dummy.h

extern "C" void setup_rust();
extern "C" void loop_rust();
// dummy.cpp

void setup() {
    setup_rust();
}

void loop() {
    loop_rust();
}
//lib.rs

#[no_mangle]
extern "C" fn setup_rust() {
    // set up in Rust
}

#[no_mangle]
extern "C" fn loop_rust() {
    // loop in Rust
}

You might wanna give your C++ files a better name than dummy, but you get the idea.

Do you have any experience with using PlatformIO dependencies from the Rust code later?

You can do this by writing your own FFI wrapper functions in C / C++ (define a function in C that calls your library, then call that function from Rust). See the Embedded Rust book on A little C with your Rust.

Alternatively you can use bindgen, and there should be examples of how to do that in the example project for this repo, but I haven't tried that out myself.

jendakol commented 2 years ago

You can do this by writing your own FFI wrapper functions

In theory, I do know it :-) I've read the documentation so I know what to expect. However, in this project, I didn't get to this point since my "hello world" is not working πŸ˜‚ So I asked more about whether you actually tried it and it is so easy as it should be.

I've used your changes but in https://github.com/jendakol/esp32-rust-pio-test I'm still getting:

jenda@blade: /home/jenda/dev/arduino/esp32-rust-pio-test [main]Γ— Β» cargo pio build
Found compatible PlatformIO Core 6.1.4 -> /home/jenda/.platformio/penv/bin/platformio
Warning! Ignore unknown configuration option `rust_lib` in section [env]
Warning! Ignore unknown configuration option `rust_target` in section [env]
Processing debug (board: esp32dev; platform: espressif32; framework: arduino)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (5.1.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.20004.220818 (2.0.4) 
 - tool-esptoolpy @ 1.30300.0 (3.3.0) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 32 compatible libraries
Scanning dependencies...
No dependencies
Building in debug mode
Compiling .pio/build/debug/src/dummy.cpp.o
Linking .pio/build/debug/firmware.elf
/home/jenda/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/debug/src/dummy.cpp.o:(.literal._Z4loopv+0x4): undefined reference to `hello_from_rust'
/home/jenda/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: .pio/build/debug/src/dummy.cpp.o: in function `loop()':
/home/jenda/dev/arduino/esp32-rust-pio-test/src/dummy.cpp:20: undefined reference to `hello_from_rust'
collect2: error: ld returned 1 exit status
*** [.pio/build/debug/firmware.elf] Error 1

Don't get me wrong, I have some experience with "normal Rust writing" (Linux-running microservices) as well as C++ (because I did write some projectsfor ESP32) so I'm not completely lost but I'm still missing some important pieces regarding wiring Rust and C++ together, not speaking about PlatformIO.

So I highly appreciate your help :-)

DanielRoeven commented 2 years ago

OK so you might be encountering a different bug which I noticed too... and if you are... I'll create an issue for this too. I thought it was just my setup that was causing issues ("it ~works~ on my machine"). Hence why I didn't mention it initially. It seems like an unrelated issue:

What's going on for me is the incremental builds of C/C++ first, then Rust, and then linking them, isn't working correctly.

But! Try this:

  1. If you get your C/C++ to work first (so leave your headers and your rust definitions, but outcomment the place where you actually call hello_from_rust(). Your code should compile, indicating that A) Your C/C++ files are correct and Arduino is being linked correctly and B) that you header file and function signature are correct.
  2. Now that your C/C++ project builds, it should also have built the Rust part (and linked, even though there is nothing to link until you actually add back the hello_from_rust()) call. (You can verify this by breaking your Rust file (invalid syntax, whatever, just see to it that your C/C++ compiles and that Rust fails). Critical is that you end on a step where both C/C++ and Rust have compiled correctly.
  3. Now add back your hello_from_rust() call in C++ and build again. This is the step where it magically started working for me. I don't know exactly what's going on, but somehow it seems like it's using the cached Rust build for the linker? So that if the whole project compiled successfully without an FFI call, then the next build, all the symbols will be in place in the cached build, and the next time you build with an FFI call, the linker doesn't complain.

It's a little strange what's going on, and I kind of have a hard time trusting or verifying my own theory (but deleting .pio/ and target every time does reliably create a non-working build the first time, which I reliably get working with the steps above). This tick-tock build cycle is of course driving me a little crazy, but who knows, we might be able to get to the bottom of it and solve the issue (which is really separate from this thread.) So I'm very curious if the above steps solve it for you, and if they do, I'll open a new issue!

Cheers!

jendakol commented 2 years ago

Before I even try it, I have to share with you that "tick-tock build cycle" is exactly what I'm experiencing. I also struggle with another error saying this:

Building in debug mode
__run_cargo([".pio/build/debug/firmware.elf"], [".pio/build/debug/src/dummy.cpp.o"])
cargo build --lib --target xtensa-esp32-espidf
   Compiling esp32-rust-pio-test v0.1.0 (/home/jenda/dev/arduino/esp32-rust-pio-test)
error[E0463]: can't find crate for `std`
  |
  = note: the `xtensa-esp32-espidf` target may not support the standard library
  = note: `std` is required by `esp32_rust_pio_test` because it does not declare `#![no_std]`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`

error: cannot find macro `println` in this scope
  --> src/lib.rs:19:5
   |
19 |     println!("hi there!");
   |     ^^^^^^^

error: `#[panic_handler]` function required, but not found

For more information about this error, try `rustc --explain E0463`.
error: could not compile `esp32-rust-pio-test` due to 3 previous errors
*** Error 101
*** [.pio/build/debug/firmware.elf] Error 101

So I think that only when this error occurs, it's actually compiling the Rust and it's ignoring it completely otherwise! (the code in repo has no STD stuff added so it doesn't suffer from this but try to comment L1-9 and uncomment L19, you'll see by yourself πŸ˜ƒ).

DanielRoeven commented 2 years ago

Lol, I think I solved that one too. Too bad you're also experiencing these problems, but on the other hand, good to see that it's not just my environment.

To solve above issues, make a new folder .cargo in your root, and a file config.toml in that.

// .cargo/config.toml

[build]
target = "xtensa-esp32-espidf"

[unstable]
build-std = ["std", "panic_abort"]
build-std-features = ["panic_immediate_abort"]

I don't even know where I got this code from. It isn't created as boilerplate with a fresh project generation anymore. I think it might have in a previous version of cargo-pio?

ivmarkov commented 2 years ago

@DanielRoeven @jendakol

First of all, some general notes on the cargo pio project's state and what you guys are trying to achieve:

General state of cargo-pio

cargo pio hasn't seen much love recently, simply because there was not a big inflow of users willing to use it. So it has accumulated bugs that I'll fix (more on that below).

Usage of cargo-pio

But in general, using cargo pio means that you are diving into an exercise, where you'll need to interface whatever C/C++ Vendor SDK you are willing to use together with Rust. Meaning, that once the "template" project builds successfully:

I would say, the above exercise is worth it only if you are very experienced with both Rust and C(++) and have a very specific use case you would like to pursue. The one exception to the above is interfacing with Espressif's ESP-IDF framework which is very well supported, but even there I suggest you abandon PlatformIO, because while it is possible with PlatformIO, it is for advanced use cases only and definitely the worst way to start. More on that below.

Alternatives?

Assuming that you are interested in coding MCUs in Rust, there are more ergonomic ways. But ALL of them require (a) abandoning PlatformIO and (b) (potentially - except for ESP-IDF) abandoning the C/C++ Vendor/Framework SDK that you know and currently use (via PlatformIO or otherwise).

The Rust Embedded community is well-developed and there are tons of libraries there, but they tend to be Rust libraries, and the overall community is Rust-centric, with little to no C code used (there is one exception w.r.t. ESP-IDF, I'll get there below).

In terms of Rust support for various MCUs, I would split the Rust community as follows:

Rust on Espressif chips (ESP32XX and to some extend - ESP8266):

You have to look into this repo: https://github.com/esp-rs and read the Rust Book listed there. The support for using Rust on Espressif chips is great, and it covers both bare-metal (no vendor SDK whatsoever), as well as ESP-IDF (but NOT Arduino!) based Rust.

None of the above options have anything to do with cargo-pio though. With or without ESP-IDF the build system is purely Rust cargo based (with options to use it from ESP-IDF's specific CMAKE build system and PlatformIO but these are advanced use cases).

Matrix room for discussions here: https://app.element.io/#/room/#esp-rs:matrix.org

Rust on other chips (primarily STM32 and NRF but I think not only!)

I would strongly suggest looking into https://github.com/embassy-rs/embassy for these. Embassy is a very strong, Rust async-based platform with good support for the cortex-m ecosystem.

Matrix room for discussions here: https://app.element.io/#/room/#embassy-rs:matrix.org

Bugs

I'll shortly release a bug-fixing new version, that will take care at least of the build issues. Will notify here and close the issue once that's done.

But then again, the above will only get you to a buildable "template" project. Interfacing Rust with C(++) will remain on the shoulders of the user.

jendakol commented 2 years ago

@ivmarkov Thanks for the explanation.

Maybe let me explain my use case, so we understand each other.

I wrote several applications for ESP32, using C++, Arduino framework, and PlatformIO. The reasons for this are:

  1. C++: A year ago, the state of Rust-on-ESP32 seemed much more complicated than now (and while I'd love to help with it, I just don't have the required skillset nor time to obtain it). However, the C++ is definitely not a voluntary choice for me (on the contrary, I just hate it).
  2. Arduino framework: It's just more convenient to use the Arduino framework than ESP IDF for someone like me, who came from Arduino boards and wants to take advantage of the widely-used ecosystem of libraries.
  3. PlatformIO: I really love that PIO creates a dependency/packaging system for embedded development, so it's very easy to add and use a 3rd party library.

What I want now is to ditch the C++ code written by me and replace it with Rust (for obvious reasons) while still using the existing libraries I'm used to (and which are written in C++). So considering my knowledge about building C/C++ code is approaching zero, I hoped cargo pio might be a way for me, expecting that:

  1. It will enable me to write my own code in Rust, using all the great features the language has.
  2. I'll still be able to use the 3rd party libraries; I will "only" be required to write the bindings for it (like I did on x86 development before). I'm afraid that writing everything again in pure Rust is too difficult and time-consuming (even "standard" Rust crates are often just a facade over some well-known and battle-proven C/C++ library).
  3. Everything will be just great.

So I definitely don't want to call Rust from C++ (that was just we tried a workaround with @DanielRoeven for something which doesn't work), only the opposite.

This being said, do you see an easy-enough way how to achieve my goals?

DanielRoeven commented 2 years ago

Thanks for the summary, @ivmarkov!

the project does NOT offer any pre-built mappings of C/C++ Vendor SDKs to Rust

In other words, interfacing Rust with C++ goes via C.

Interfacing Rust with C(++) will remain on the shoulders of the user.

No confusion there! The way I understand the ecosystem, this seems entirely reasonable and obviously not this project's goal. It's absolutely fine (even if laborious and annoying) to interface with C++ libraries through extern "C" interfaces that we write ourselves.

My initial confusion stemmed from the fact that there were Arduino setup() and loop() functions defined in the Rust "template" project. These functions don't work and never will, as I surmised and you just explained πŸ™‚

I would say, the above exercise is worth it only if you are very experienced with both Rust and C(++) and have a very specific use case you would like to pursue.

Not too experienced with Rust or C++ but I do have a specific use case (want PIO + a specific Arduino lib + Rust), so I'm willing to jump through the hoops required. So far it's gone pretty well, thanks to your cargo-pio project, so thanks for that!

I'll shortly release a bug-fixing new version, that will take care at least of the build issues. Will notify here and close the issue once that's done.

Fantastic. Let me know if there's anything I can do to help β€” like I wrote in the initial post, I've gotten it all working, but it required a non-trivial amount of reworking the template project, where it's easy to be lead astray by false hints πŸ™‚

To try and recap the bugs in this thread:

  1. The setup() and loop() stubs in lib.rs can't and won't work and are confusing for new users.
  2. KeyError : 'framework-arduino' from platformio.cargo.py when generating a project with the Arduino framework. Correct name for said library is framework-arduinoespressif32 in most (all?) cases. (Or maybe a lookup using board + platform + desired library should be done?)
  3. The linker failing to find Rust / C FFI function definitions, based on the "tick-tock" build cycle, as described in https://github.com/esp-rs/embuild/issues/60#issuecomment-1222076717. I'm fairly certain this is a regression, as older versions of cargo-pio did not suffer from this problem.
  4. .cargo/config.toml is not included in the template project anymore, but it seems to be required for the Rust std lib to work.

Thanks again, and let me know how / if I can help out.

jendakol commented 2 years ago

Lol, I think I solved that one too. Too bad you're also experiencing these problems, but on the other hand, good to see that it's not just my environment.

To solve above issues, make a new folder .cargo in your root, and a file config.toml in that.

// .cargo/config.toml

[build]
target = "xtensa-esp32-espidf"

[unstable]
build-std = ["std", "panic_abort"]
build-std-features = ["panic_immediate_abort"]

I don't even know where I got this code from. It isn't created as boilerplate with a fresh project generation anymore. I think it might have in a previous version of cargo-pio?

Interesting, I have this file/config there - only it was sayin' build-std = ["core", "panic_abort"] and not build-std = ["std", "panic_abort"] πŸ˜„ (so, pretty easy fix if you know where to touch...)

Anyway, I've just succeeded to run my Rust code (Hello World version) on the ESP32. Even though it wasn't nor easy nor clean, it works. (updated the repo) Thanks!!

Now let's see what will @ivmarkov prepare as a fix - many thanks to you both, guys. I really appreciate it!

ivmarkov commented 2 years ago

What I want now is to ditch the C++ code written by me and replace it with Rust (for obvious reasons) while still using the existing libraries I'm used to (and which are written in C++). So considering my knowledge about building C/C++ code is approaching zero, I hoped cargo pio might be a way for me, expecting that:

  1. It will enable me to write my own code in Rust, using all the great features the language has.
  2. I'll still be able to use the 3rd party libraries; I will "only" be required to write the bindings for it (like I did on x86 development before). I'm afraid that writing everything again in pure Rust is too difficult and time-consuming (even "standard" Rust crates are often just a facade over some well-known and battle-proven C/C++ library).

You are grossly underestimating the maintenance effort necessary to "only" write bindings for C/C++ libraries and call them from there. You might also be underestimating the amount of C libraries you need for embedded development with Rust - especially for the Espressif chips.

Unless the C libraries you use are some sort of domain-specific computational stuff, I would say it is not worth it to invest your effort in interfacing with those, and instead you should search for Rust equivalents. Especially if you plan to use - from Rust - basic stuff like MCU drivers, wifi, web server etc where these are coded in C/C++.

So I definitely don't want to call Rust from C++ (that was just we tried a workaround with @DanielRoeven for something which doesn't work), only the opposite.

But the other direction (calling C++ from Rust) is not easy at all either, as I hopefully explained.

(even "standard" Rust crates are often just a facade over some well-known and battle-proven C/C++ library).

Sure. But: a) these facades take a lot of effort to maintain. I speak from my own experience, as I'm maintaining the esp-idf-hal and esp-idf-svc facades to ESP-IDF. b) In embedded Rust, maintaining these facades is more rare than with regular OSes. ESP-IDF is one exception to that, but is one of only a few.

This being said, do you see an easy-enough way how to achieve my goals?

Not really. You are between a rock and a hard place. Especially with Arduino. If it was a pure C SDK, kind of fine. But with C++ - not worth it, IMO.

ivmarkov commented 2 years ago

I would say, the above exercise is worth it only if you are very experienced with both Rust and C(++) and have a very specific use case you would like to pursue.

Not too experienced with Rust or C++ but I do have a specific use case (want PIO + a specific Arduino lib + Rust), so I'm willing to jump through the hoops required. So far it's gone pretty well, thanks to your cargo-pio project, so thanks for that!

Well up to you and to be honest - I started exactly with your mindset as well, but reality has taught me otherwise - in other words, start with Rust, leave behind the PlatformIO and Arduinos of this world, try to find replacement for the magnificent Arduino lib in Rust, and only if you absolutely cannot find replacement, think about interfacing with this lib. Espoecially if the lib is in C++ which seems to be the case.

May I ask what lib we are talking about?

To try and recap the bugs in this thread:

  1. The setup() and loop() stubs in lib.rs can't and won't work and are confusing for new users.

Will be fixed.

  1. KeyError : 'framework-arduino' from platformio.cargo.py when generating a project with the Arduino framework. Correct name for said library is framework-arduinoespressif32 in most (all?) cases. (Or maybe a lookup using board + platform + desired library should be done?)

Will be fixed.

  1. The linker failing to find Rust / C FFI function definitions, based on the "tick-tock" build cycle, as described in cargo-pio PlatformIO-first build with Arduino SDKΒ #60 (comment). I'm fairly certain this is a regression, as older versions of cargo-pio did not suffer from this problem.

I'm pretty sure this is a regression in recent versions of PlatformIO, not in cargo-pio. It seems that adding a Python build hook on the projenv no longer works reliably, and it should be done on the env instead. However, this has other difficulties further down the path, but for now that's what I'll do.

  1. .cargo/config.toml is not included in the template project anymore, but it seems to be required for the Rust std lib to work.

Nope, this I won't do. Folks who would like to switch to STD will have to add this file themselves. Or just enumerate these options as command-line parameters to cargo in platformio.ini. Basically: cargo_options = -Zbuild-std="std,panic_abort"

(the build_std_features thing is no longer necessary with ESP-IDF 4.4+ which is now supported by PIO)

Thanks again, and let me know how / if I can help out.

I'll notify you once the new cargo-pio is released, so that you can check whether the fixes work for you.

DanielRoeven commented 2 years ago

Well up to you and to be honest - I started exactly with your mindset as well, but reality has taught me otherwise - in other words, start with Rust, leave behind the PlatformIO and Arduinos of this world, try to find replacement for the magnificent Arduino lib in Rust, and only if you absolutely cannot find replacement, think about interfacing with this lib. Espoecially if the lib is in C++ which seems to be the case.

Understood! I'll defer to your judgment, and increase the priority of switching over to Rust full-time. In the meantime I'll continue hacking with cargo-pio, but I understand what you're saying. Like @jendakol it's always been my preferred option to go full-time Rust, but it seemed too laborious to convert all of my PIO + Arduino projects to pure Rust.

May I ask what lib we are talking about?

Among various libs that I've built various Arduino projects on, ... primarily WiFi. Searching for Rust+ESP32 WiFi modules gave this: https://github.com/esp-rs/esp-wifi, but it has Warning: experimental ⚠️ written all over it. And it's the most "mature" Rust + ESP WiFi lib that I found. Whereas Arduino's WiFi stack "just works", even through the cumbersome writing of bindings.

Or maybe I am missing something? What would you recommend for WiFi on ESP + Rust?

Nope, this I won't do. Folks who would like to switch to STD will have to add this file themselves.

Gotcha. I might have remembered incorrectly that it came with the template project previously. Maybe just a mention in the README would suffice.

Cheers!

jendakol commented 2 years ago

"You are between a rock and a hard place."

😐 Yeah, seems like that. That's unfortunate... Here's an example list of which libraries I need to use:

adafruit/Adafruit NeoPixel
robtillaart/PCF8574
bodmer/TFT_eSPI
adafruit/RTClib
earlephilhower/ESP8266Audio
pablo-sampaio/Easy MFRC522
painlessmesh/painlessMesh

For all the others (AsyncTCP, Wifi, ESPAsyncWebServer, ArduinoJSON, ...) I believe I can find more than suitable replacements in the Rust ecosystem, considering I have std available, so everything should kind-of work (Is it that case? Is it that easy?). But for these mentioned above, these are sth like drivers for a specific HW (displays, RTC module, NFC tag reader, ...), and I just can't imagine I'm rewriting everything into pure Rust. Especially the last one is kind of special as it uses the ESP32 Wi-Fi module to do some magic and work like a mesh network (really cool stuff!).

Maybe I imagine it too simply, but... I thought that PIO would compile everything (meaning C++ libs) together like now, and all I'd have to do is to write bindings for the libs API, only the methods I really call and nothing else. That couldn't be that hard, no? 😰

ivmarkov commented 2 years ago

Well up to you and to be honest - I started exactly with your mindset as well, but reality has taught me otherwise - in other words, start with Rust, leave behind the PlatformIO and Arduinos of this world, try to find replacement for the magnificent Arduino lib in Rust, and only if you absolutely cannot find replacement, think about interfacing with this lib. Espoecially if the lib is in C++ which seems to be the case.

Understood! I'll defer to your judgment, and increase the priority of switching over to Rust full-time. In the meantime I'll continue hacking with cargo-pio, but I understand what you're saying. Like @jendakol it's always been my preferred option to go full-time Rust, but it seemed too laborious to convert all of my PIO + Arduino projects to pure Rust.

May I ask what lib we are talking about?

Among various libs that I've built various Arduino projects on, ... primarily WiFi. Searching for Rust+ESP32 WiFi modules gave this: https://github.com/esp-rs/esp-wifi, but it has Warning: experimental ⚠️ written all over it. And it's the most "mature" Rust + ESP WiFi lib that I found. Whereas Arduino's WiFi stack "just works", even through the cumbersome writing of bindings.

Please check the ESP-RS Rust Book - this topic is covered there. In short, you are looking at support for Wifi with Rust baremetal, which is indeed still WIP. Wifi support on top of ESP-IDF (but then again, ESP-IDF C APIs are all hidden from you and nicely dressed with safe Rust) is working since at least an year, if not two.

Or maybe I am missing something? What would you recommend for WiFi on ESP + Rust?

I strongly suggest looking at:

The former is a template generator similar to cargo-pio, but specific for Espressif chips & platform, and requires only pure Rust.

The latter is a small demo I did a few years ago to show the capabilities of using pure rust on top of ESP-IDF.

ivmarkov commented 2 years ago

"You are between a rock and a hard place."

😐 Yeah, seems like that. That's unfortunate... Here's an example list of which libraries I need to use:

adafruit/Adafruit NeoPixel
robtillaart/PCF8574
bodmer/TFT_eSPI
adafruit/RTClib
earlephilhower/ESP8266Audio
pablo-sampaio/Easy MFRC522
painlessmesh/painlessMesh

For all the others (AsyncTCP, Wifi, ESPAsyncWebServer, ArduinoJSON, ...) I believe I can find more than suitable replacements in the Rust ecosystem, considering I have std available, so everything should kind-of work (Is it that case? Is it that easy?).

Mostly yes, especially if you work on top of ESP-IDF (STD). See my previous post how to get started with it.

As for the others, I don't have the time to look deeply into these, but just some pointers to a few:

bodmer/TFT_eSPI

embedded-graphics + a Rust driver crate for whatever HW your display is using. My rust-esp32-std-demo crate has at least 4 demos of precisely embedded-graphics + HW specific Rust drivers.

earlephilhower/ESP8266Audio

I have to check what the i2s support is in esp-idf-hal, but if it is still missing, it won't be easy to add it, as it is of course available in ESP-IDF

adafruit/RTClib

What are you using this for? Seems strange together with an Espressif chip?

adafruit/RTClib

https://crates.io/crates/rfid-rs/0.1.2

BTW forgot to mention: if you are still on ESP8266... Rust support for ESP8266 is weak (as in no Wifi and lagging behind). You should really switch to ESP32XX if possible.

ivmarkov commented 2 years ago

@DanielRoeven @jendakol I've released a bugfix for cargo-pio. If you could try upgrading (cargo install cargo-pio) and then generating a PIO<->Rust project again...

ivmarkov commented 2 years ago

Maybe I imagine it too simply, but... I thought that PIO would compile everything (meaning C++ libs) together like now, and all I'd have to do is to write bindings for the libs API, only the methods I really call and nothing else. That couldn't be that hard, no? 😰

It is quite hard, with C++. How do you call a method of a C++ class from Rust?

DanielRoeven commented 2 years ago

@ivmarkov the bug fix works beautifully β€” no more need for the workarounds described above. Thanks for the effort! I'll close this issue.

jendakol commented 2 years ago

@ivmarkov OK, you got me πŸ˜„ Truth is I had so shallow knowledge of this I considered calling C++ classes/methods from Rust a solved problem. I'll have to look at the cxx.rs, that looks promising...
Anyway, at the same time, we're (me and my colleague I work on projects with) evaluating the possibility to ditch not only C++ but also the Arduino framework which leaves us with raw esp-idf-hal and esp-idf-svc crates. We'll let you know if you're interested (maybe even if not, for any possible future readers of this thread πŸ˜„).

BTW, for the libraries:

  1. rfid-rs, embedded-graphics - will try, thx.
  2. "I have to check what the i2s support is in esp-idf-hal, but if it is still missing, it won't be easy to add it, as it is of course available in ESP-IDF" - it WILL or WILL NOT?
  3. RTC lib - we needed persistence of the time during the device shutdown :-)

Thanks!