How do we include existing C++ libraries

[trezm]

I love this project, but I want to use the Adafruit NeoPixel library. I'm happy to start digging in to the work, I'm just not sure where to start. Can someone point me in the right direction?

[japaric]

I love this project

❤️

Can someone point me in the right direction?

To bind to C/C++ libraries the approach is to:

• read the FFI documentation :-)

• create a new crate (cargo new --lib foo-sys)

• in the build script of that crate use the gcc, the cmake or the make-cmd crate to build the C/C++ library from source code.

  • Alternatively you could build the C/C++ library by hand and just ship the libfoo.a file with the crate provided that a proper license is used. If you go this route you'll have to use a build script to copy libfoo.o somewhere the top dependency can find it ($OUT_DIR), and also print a cargo:rustc-link-lib=static=foo line and a cargo:rustc-link-search=native=$OUT_DIR line from it as well.
  • Make sure you test this for the right target! xargo build --target thumbv7m-none-eabi

• use the bindgen tool to create bindings to the C/C++ library

  • for starters you can just put all the generated code in src/lib.rs
  • eventually you'll want to put the raw bindings in src/ll.rs or similar and expose safe abstractions build on top of those bindings in src/lib.rs. You can use photon-hal as a reference for this step.

• in your photon-quickstart project add foo-sys as a dependency and test it.

Let me know if run into any problem.

[trezm]

Fantastic overview!!! I got it working very easily with a simple library, but it raised a second question:

I'm specifically trying to get the adafruit neopixel library working, which has dependencies on WProgram.h and Arduino.h, how should I go about including these in the build process?

[japaric]

I'm specifically trying to get the adafruit neopixel library working, which has dependencies on WProgram.h and Arduino.h, how should I go about including these in the build process?

And those are in the spark/firmware source code. I'd say just copy them into your neopixel crate for now (make sure to get the ones from v0.6.2) and pass -I$CARGO_MANIFEST_DIR/user/inc to gcc / cmake in the build script. Ideally this should be "generic" so that the neopixel crate can be compiled for different boards / devices but I don't think you should bother with that right now; I'd suggest you focus on getting this working with the photon first and revisit the generic issue later.

[trezm]

Still working through it -- getting the following error now:

/Users/pete/dev/particle/firmware/hal/inc/pinmap_hal.h:73:5: error: user-defined literal in preprocessor expression

when my build looks like:

   let firmware_dir = "/Users/pete/dev/particle/firmware";
    gcc::Config::new()
        .cpp(false) // Switch to C++ library compilation.
        .compiler("arm-none-eabi-gcc")
        .target("thumbv7m-none-eabi")
        .flag(&format!("-I{}/{}", firmware_dir, "inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "user/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "wiring/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "system/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "services/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "communication/src"))
        .flag(&format!("-I{}/{}", firmware_dir, "hal/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "hal/shared"))
        .flag(&format!("-I{}/{}", firmware_dir, "hal/src/core"))
        .flag(&format!("-I{}/{}", firmware_dir, "hal/src/stm32"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/shared/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/MCU/STM32F1xx/CMSIS/Include"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/MCU/STM32F1xx/CMSIS/Include/core_cmFunc.h"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/MCU/STM32F1xx/SPARK_Firmware_Driver/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/MCU/shared/STM32/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/MCU/STM32F1xx/STM32_StdPeriph_Driver/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/MCU/STM32F1xx/STM32_USB_Device_Driver/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "platform/NET/CC3000/CC3000_Host_Driver"))
        .flag(&format!("-I{}/{}", firmware_dir, "dynalib/inc"))
        .flag(&format!("-I{}/{}", firmware_dir, "applications/blink"))
        .flag(&format!("-I{}/{}", firmware_dir, "libraries"))

        .flag("-DSTM32_DEVICE")
        .flag("-DSTM32F10X_MD")
        .flag("-DPLATFORM_THREADING=0")
        .flag("-DPLATFORM_ID=0_SPARK=0x1D50")
        .flag("-DUSBD_PID_DFU=0x607F")
        .flag("-DUSBD_PID_CDC=0x607D")
        .flag("-DSPARK_PLATFORM")
        .flag("-DFLASHEE_Ertex-m3")
        .flag("-mthumb")
        .flag("-flto")
        .flag("-DINCLUDE_PLATFORM=1")
        .flag("-DPRODUCT_ID=0")
        .flag("-DPRODUCT_FIRMWARE_VERSION=6U_BUILD_ENABLE")
        .flag("-DSYSTEM_VERSION_STRING=0.6.2")
        .flag("-DRELEASE_BUILD")
        .flag("-std=gnu++11")

        .file("src-cpp/neopixel.cpp")
        .compile("libneopixel.a");

(I basically took the firmware's compilation step outputted by make PLATFORM=photon APP=blink and copied those over to the build step.)

Any ideas?

[japaric]

hmm, I think this may not be correct:

        .flag("-DPLATFORM_ID=0_SPARK=0x1D50")

Definitions, -D flags, should have one equal sign in them or none. Maybe this is actually two definitions? -DPLATFORM_ID=0 and -DSPARK=0x1D50? I'm not familiar with the internals of spark/firmware build system so I can't say for sure.

[trezm]

Alright, I think I've figured this all out, I have to run out right now, but I didn't want you to think I'd forgotten after you've given so much help!

Once I verify I'll write both a short blog post and a summary in this thread.

[japaric]

@trezm Excellent! Happy to do a code review / provide suggestions if you want.

[trezm]

So close, so the crate compiles properly on it's own, but when included in the photon-quickstart's cargo file, and run xargo build I get the following error:

warning: warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: warning for library: /Users/pete/dev/particle/photon-quickstart/target/photon/debug/build/neopixel_rs-e975353f7f01d837/out/libneopixel.a the table of contents is empty (no object file members in the library define global symbols)
error[E0463]: can't find crate for `std`
  |
  = note: the `photon` target may not be installed

error: aborting due to previous error(s)

Not sure what the issue is here! Something to do with using xargo perhaps? I checked and am using the nightly version of rust as well.

Edit: Forgot to link to the repo for the crate here

[trezm]

Just had a thought and tried compiling neopixel_rs with xargo as well, got a slew of errors,

error[E0463]: can't find crate for `std`

error: aborting due to previous error(s)

error: Could not compile `itoa`.
Build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error(s)

error: Could not compile `gcc`.
Build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error(s)

error: Could not compile `libc`.
Build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error(s)

error: Could not compile `serde`.

I'll keep working on it though!

[trezm]

Got a little farther (I think), adding your steed library makes those warnings go away, but now I'm getting similar errors about platform::*;

W
error[E0432]: unresolved import `platform::*`
  --> /Users/pete/.cargo/registry/src/github.com-1ecc6299db9ec823/sc-0.1.5/src/lib.rs:21:9
   |
21 | pub use platform::*;
   |         ^^^^^^^^^^^^ Maybe a missing `extern crate platform;`?

so... getting closer?

[japaric]

@trezm The crate needs to have a #![no_std] in it (at the top of src/lib.rs -- like this). That indicates that it doesn't depend on the std crate (the standard library). This is required because the std crate doesn't support ARM Cortex-M targets like the photon.

adding your steed library

steed is a re-implementation of std that targets devices running Linux so it won't work in this case.

[protomors]

@trezm I would also like to hear about your results. There are many existing libraries written in C so your experience will be useful.

But maybe in this case it is easier to just implement protocol for this LED strips from scratch? I did not work with them but according to datasheet for LEDs protocol is very simple (different pulse width corresponds to 0 or 1). On STM32 it is possible to just bit-bang it. Even easier way to generate series of impulses with precise timing is to use USART configured for specific speed.

[trezm]

Alright -- I think I have everything working now, I just have to test with an actual strip of lights which, unfortunately, I won't have access to for the next 2 days or so :(.

That being said, everything compiles! I'm hoping it's just that easy.

@protomors My initial thought was actually to replicate the logic in Rust, but it looked like there were a lot of optimizations that they did and I wasn't sure whether they were required/necessary. Instead of going down that rabbit hole and replicating their work, I was hoping to be able to pound out a more general form that we could use for any Arduino C lib -> Rust lib

That being said, if this doesn't work I might scrap the whole thing and do it manually :)

[protomors]

@trezm I looked into this NeoPixel library and implementation is indeed very complex. But the main reason for it is that they support different target devices (even AVR at low frequencies). So most implementations had to be written in assembly. But if you are using Cortex-M microcontroller everything is much easier. USART+DMA is the best approach (in the past I used it to write library for 1-wire protocol). At high enough system clock frequency you can simply use GPIO+Delays (but it will have to block execution of any other task for correct timings).

In any case your efforts will not be wasted. It will be nice to have a way to use any Arduino library in rust project.

[trezm]

@protomors ahh, that makes sense! I saw the assembly and noped out of there very quickly. I might revisit it regardless to see if I can get it to run so I don't need unsafe blocks.

[trezm]

Unfortunately, the library fails when you try to run it :(. Specifically the ::show() call, I'm thinking it's beyond my help at this point -- but I'll start writing a quick guide for what I did to compile a c library!

[japaric]

@trezm :-(. Can you say more about how the library is failing? Are you hitting an exception or similar?

[trezm]

Unfortunately that's all the info I have :-( -- I load it on to the device and as soon as I call the .show() method, I get the blinking red light of death and it reboots.

[protomors]

@trezm I did not worked with particle platform. But if I understand right - it uses special bootloader and you are using it to flash firmware. Isn't there a way to connect debugger to a device?

[trezm]

@protomors I'm not sure to be honest, but if there is I'd be happy to use it and report back!

[protomors]

@trezm I found out that red blinking are also indicating the type of error. https://docs.particle.io/guide/getting-started/modes/photon/#red-flash-sos So you could try to decode it. But errors themselves are pretty generic so I don't know if they will help to find exact reason why your program is failing.

And about your original task. I can help you to implement protocol for controlling this LED strips in rust. But I have no idea how to integrate it with Particle platform.

[trezm]

I gave implementing it in rust a shot last night, unfortunately rust isn't the hard part here, it's dealing with a clock and platform where the most granular time unit exposed is micros, yet the neopixel needs nanos. I was able to implement a basic protocol last night, but the timing is incorrect so the color is not quite right. I have an oscilloscope on its way and should be able to further test what's going on when it arrives.

My suspicion though is that I'm not using fastSet in the particle library and using the safe version (digitalWrite) which takes significantly longer.

[protomors]

The other reason for imprecise delays are, probably, some background tasks that are interrupting you program. That is why I think that you will have the best chance of success by using USART. You will need to set baud rate to 2400000. This will transmit 3 bits in exactly 1.25 microseconds. Then you encode data for you LEDs using sequence 100 to transmit 0 and 110 to transmit 1. I hope you understand how this works. It is hard to explain without pictures. After preparing complete sequence you transmit it all at once using USART.

[trezm]

I believe the max baud rate is only 115200, so that might not be an option. In addition the spec says that the highs and lows are not necessarily divided into three even chunks, so I'm not 100% sure that timing would work anyway.

I also would like to be able to use any arbitrary pin, not necessarily just the serial pins, designated as Tx/Rx or 28/29 according to the docs.

I'm attempting to use what the original library did now, there are methods within the particle's assembly target that allow turning on/off interuptions -- namely:

__ASM volatile ("cpsie i" : : : "memory");
// and
__ASM volatile ("cpsid i" : : : "memory");

So I'm including those as assembly in the rust implementation as well. I believe the large missing piece is simply the fast access to pins here, which I am currently working on.

[japaric]

@trezm If you have access to PWM you can try this method. If you also have access to DMA you can have the process running in the background. If you don't have access to DMA you should still be able to use this method manually by changing the duty cycle in a loop -- I think you can reduce the frequency by quite a bit if the recommended 400 KHz value is too high as long as you keep the on times unchanged.

[trezm]

@japaric Verrrrrry interesting, I'll give that a shot if getting setPin doesn't work out!