WIP: Esp32 support

[xermicus]

Run sled on ESP32 boards.

This is still WIP!

[xermicus]

Cool, thank you much for the thorough review! It's really WIP and your corrections/suggestion are very welcome :-) I'll later try to add sled to platformio as a library. This would mean you can build it with make without having to worry about deps, and later use it as a library. platformio will keep track of all freertos and other dependencies and also link it correctly. This way it will be very easy for others to use.

[vifino]

Also, any generic building things, like linking into a library, should be in the main GNUmakefile, esp32.GNUmakefile should contain only esp32 specific overrides.

By the way, you don't have to implement all of these, most of them are suggestions and if the need arises to implement them, all of the above I can do myself. Just lack some testing hw at the moment. :)

[orithena]

@vifino Dangit, if I knew this last sunday... I could've given you an ESP32.

[vifino]

@orithena Heh, I should have an ESP32 somewhere, but right now I lack 5V PSU's and ws2812b matrices that don't suck.

.. Actually, I still have a tshirt I've messed with..

[vifino]

Since platformio probably has support for the esp8266 - which is also xtensa i believe - and can also run freertos, it might be pretty simple to add support for it as well.

The output module probably has to be replaced, but that's little effort in comparison to figuring out how to get it to compile. See #26 for my failed efforts.

I'm at the hackerspace in half an hour, I got esp8266's there but probably no esp32s. I'm gonna try getting this PR to work on a small matrix there.

[xermicus]

Yes just install platformio and then platformio platform install espressif8266. Also I see a esp8266-freertos-sdk framework when I do platformio platform frameworks so everything you need should be provided. Not sure how to install the esp8266-freertos-sdk framework from the command line though but you can probably just use a dummy project to let the dependency finder install the framework...

[vifino]

Hey, if you could write some (hopefully detailed) instructions on how to build this for ESP32's, that'd be nice. Since I'm home now, I got ws2812b matrices and esp32's, so testing this would be cool. It'd also help me with doing CI.

[xermicus]

Sure, I'm working on a Dockerfile for our project where we need seld. Ill attach it here when ready.

[vifino]

Alright, cool.

[xermicus]

you can build it with pio now:

FROM archlinux/base

RUN pacman -Sy
RUN pacman -Syyu --noconfirm
RUN pacman -S --noconfirm git sudo binutils coreutils python2 python2-pip tar file awk fakeroot make 
RUN useradd -m build
RUN echo "build ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

USER build
WORKDIR /home/build
RUN git clone https://aur.archlinux.org/python-click-5.1.git
RUN git clone https://aur.archlinux.org/platformio.git
RUN git clone --single-branch --branch esp32-support https://github.com/xermicus/sled.git
RUN git clone --single-branch --branch platformio-library https://github.com/xermicus/ESP32-Digital-RGB-LED-Drivers.git

RUN cd python-click-5.1 && makepkg -si --noconfirm
RUN cd platformio && makepkg -si --noconfirm

RUN pio platform install espressif32 --with-package framework-arduinoespressif32
RUN pio lib --global install /home/build/ESP32-Digital-RGB-LED-Drivers/

RUN cd sled && \
    cp Makefiles/sledconf.esp32 sledconf && \
    make libsled.a

[vifino]

Nice, I'll add a CI config once I'm home from work. How do I get to the final binary which I can flash and use?

Pretty sure there is a bunch of support code needed. If you want, you can get a Git repository here in ShinyBlink, it'd make it easier to look for and would allow multiple people to bring in their improvements, just like in sled.

[xermicus]

Cool! No worries I'll fix all the minor things that I haven't yet. As for the firmware build I can look into it after work. Do you prefer a makefile target to build something flashable or completely depend on platformio? Doing it via a platformio project would be very easy but I think both is possible, I'll probably try both. I'll also check how easy it would be to add sled to platformio registry to see whether a separate repository is necessary.

[xermicus]

Removed the filter in libsled.a target, works fine now. Was probably due to some stray .o files laying around or something.

[xermicus]

Switched the outmodule to out_esp32_digitalledlib.c and included a check in the esp32.GNUmakefile to use xtensa toolchain if an esp32 outmodule is detected. With this change I can now successfully build the target sled (and libsled.a still working too) with freertos as os and esp32_digitalledlib as out mod. I also moved the libsled.a target to main GNUmakefile because I think it might be useful for other archs too. Also fixed/cleaned some other stuff so most your review should be implemented now :tada: The tasks/signal are still missing. But I acutally need signal in another project so chances are good I'll look into it later on :-)

[xermicus]

thx for the feedback :-) I'll look into the signals this weekend, and then rebase for the merge

[vifino]

Sounds good!

By the way, this only builds a library. How do I get something I can flash to an esp32? I've found a 64x8 matrix and an esp32.

[xermicus]

Cool! To get a flashable binary the easiest options are to use something like esp-idf or platformio. It should be doable with esp-idf but I use platformio. Maybe it is possible to add a firmware target to the esp32 Makefile but I don't know how to it... The following works for me (use pio run --target upload in the last line to directly flash it):

FROM archlinux/base

RUN pacman -Sy
RUN pacman -Syyu --noconfirm
RUN pacman -S --noconfirm git sudo binutils coreutils python2 python2-pip tar file awk fakeroot make 
RUN useradd -m build
RUN echo "build ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

USER build
WORKDIR /home/build
RUN git clone https://aur.archlinux.org/python-click-5.1.git
RUN git clone https://aur.archlinux.org/platformio.git
RUN git clone --single-branch --branch esp32-support https://github.com/xermicus/sled.git
RUN git clone --single-branch --branch platformio-library https://github.com/xermicus/ESP32-Digital-RGB-LED-Drivers.git

RUN cd python-click-5.1 && makepkg -si --noconfirm
RUN cd platformio && makepkg -si --noconfirm

RUN pio platform install espressif32 --with-package framework-arduinoespressif32
RUN pio lib --global install /home/build/ESP32-Digital-RGB-LED-Drivers/

RUN cd sled && \
    cp Makefiles/sledconf.esp32 sledconf && \
    make libsled.a

RUN mkdir sled_firmware && cd sled_firmware && \
    pio init && \
    echo $'[env:mhetesp32devkit] \n\
platform = espressif32 \n\
board = mhetesp32devkit \n\
framework = arduino \n\
lib_deps = /home/build/ESP32-Digital-RGB-LED-Drivers \n\
build_flags = -L/home/build/sled -lsled' > platformio.ini && \
    echo $'#include <Arduino.h> \n\
#include "esp_task_wdt.h" \n\
#include "FreeRTOS.h" \n\
extern "C" { \n\
    void user_main(void* pvParameters, int core); \n\
    int signal(int sign, void * hnd) { \n\
        return 0; \n\
     } \n\
} \n\
void setup() { \n\
    int pin = 22; \n\
    pinMode(pin, OUTPUT); \n\
    digitalWrite(pin, LOW); \n\
    user_main(NULL, -1); \n\
} \n\
void loop(){}' > src/main.cpp && \
    pio run

[xermicus]

Gave the event tasks a quick shot. The animations are running really smooth now on my matrix, gfx_mandelbrot (which uses taskpool?) is now working. The tasks handle is used as oscore_event. oscore_event_wait_until() blocks until an it gets unblocked by another task calling calling oscore_event_signal() using it's taskhandle or desirec_usec is reached. Can you please have a look at this? I think this might be an easy way to implement it like in os_unix but maybe I'm overlooking/misunderstanding something.

[vifino]

I don't think using the task handle as the event handle is okay. I think it'll explode sooner or later, given that the task it's running on is likely the main thread and sharing one event for all things will not really work/lead to problems.

I have hardware to test this PR out on, I'll do it when I get back from work. For now, though, I have to switch trains, talk to you later.

[vifino]

Hey, seems like I finally returned from work. I am running the port on the original sled matrix. The first I built.

Unfortunately it was running way too fast, practically like the wait has no effect. I fixed it, now I need to figure out why the watchdog kills it sometimes.

By the way, can I have commit rights to your branch/fork?

[orithena]

@vifino: the watchdog reset may be due to not running WiFi. If I use WiFi.forceSleepBegin in standard Arduino, I also have to add wdt_reset() in the loop to "manually" reset the watchdog timer. Maybe that's the point?

[vifino]

Hmm. I have changed the timers to use the Semaphore API, but while debugging why it always takes the early exit I found out that it gets called with a desired_usec of 1. No clue why. For the record, it works, but is busy-waiting.

@orithena I fixed it already, but thanks. I didn't end up needing to manually reset it. :)

[xermicus]

Hmm. I have changed the timers to use the Semaphore API, but while debugging why it always takes the early exit I found out that it gets called with a desired_usec of 1. No clue why. For the record, it works, but is busy-waiting.

Yeah I noticed that too but couldn't find out why. Sorry I got to pass some exams this week, so until the weekend I won't be able to do anything here anyways so feel free if you want :+1:

[vifino]

I noticed the original author can push in forks regardless, but thanks!

Education goes first, don't worry. I'll take a look, maybe @orithena too. After all, he does more with ESP32's.

[vifino]

BTW: The commit I pushed broke threads, but it is wrong otherwise. We need to have multiple events created from the same thread.

I need to do more debugging there, but help would be greatly appreciated. Weird stuff.

[xermicus]

Looks better now

[vifino]

Do timer and threads work now?

[xermicus]

mandelbrot2 seems to work properly on my chip but I don't have a matrix atm are there any tests for this?

[vifino]

Tests? ;)

[xermicus]

@vifino ping - I'd like to get this merged since we use this code on multiple matrices :-D

[vifino]

Same here! Perhaps it is time to just merge it. Are you aware of any remaining issues? Like the timing still being off or something.

I'll just merge it now and we'll work on the remaining issues on master.

[xermicus]

Same here! Perhaps it is time to just merge it. Are you aware of any remaining issues? Like the timing still being off or something.

I'll just merge it now and we'll work on the remaining issues on master.

:heart:

Awesome! Right now all the initial issues are gone, it works beautifully here on two different displays. If you find something off just let me know :-)