esp82XX-basic-LMIC (LoRa stack) w/ esp_nonos_sdk

[brainstorm]

Thanks @cnlohr for letting me know about this over twitter earlier:

With esp_nonos_sdk I went from:

`iram1_0_seg' overflowed by 5988 bytes

To:

`iram1_0_seg' overflowed by 5624 bytes

So still a ton of room for improvement from my naive attempt to rip apart the Arduino bits from @things4u LMIC port.

I guess I should figure out how to use the ESP builtin AES support instead of the custom one among other optimizations.

Or @projectgus, perhaps you know of a small/efficient LoRa/RFM95 lib for the esp8266 that I'm not aware of?

[cnlohr]

Are you using the .ld linker script that comes with esp82xx? It should give you a good bit more room. https://github.com/cnlohr/esp82xx/blob/master/ld/linkerscript.ld

Another thing to consider is many parts of the LoRa interface should be able to be put into IROM, freeing up the IRAM.

I will agree, though. There are a few specific things in the nonos sdk I really wish I could stick into IROM. Several things that only happen when you're creating new sockets and stuff like that.

Alas - the only option may be freertos.

[brainstorm]

Hey! Yes, that's the linkerscript.ld we discussed about a couple of weeks ago, seems to be in there too on its latest version, together with the eagle one:

/home/romanvg/esp8266/esp-open-sdk/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc -mlongcalls 
-Os -I/home/romanvg/esp8266/esp_nonos_sdk/include -Iesp82xx/include -I. -Iuser/lmic 
-Iesp82xx/fwsrc -Iuser -DICACHE_FLASH -DCFG_eu868 -DCFG_DEBUG -DCFG_sx1272_radio 
-DCFG_BARE_LORA_ESP82XX -DWEB_PORT=80 -DCOM_PORT=7777 
-DBACKEND_PORT=7878  -DSLOWTICK_MS=50 
-DVERSSTR='"Version: v0.1-12-gfa6c7-dev - Build Tue, 14 Feb 2017 08:23:10 +0100 with -DICACHE_FLASH -DCFG_eu868 -DCFG_DEBUG -DCFG_sx1272_radio 
-DCFG_BARE_LORA_ESP82XX -DWEB_PORT=80 -DCOM_PORT=7777 
-DBACKEND_PORT=7878  -DSLOWTICK_MS=50"' user/lmic/lmic.c user/lmic/oslmic.c 
user/lmic/radio.c user/lmic/hal.c user/lmic/aes.c esp82xx/fwsrc/uart.c 
esp82xx/fwsrc/esp82xxutil.c esp82xx/fwsrc/flash_rewriter.c 
esp82xx/fwsrc/http.c esp82xx/fwsrc/commonservices.c esp82xx/fwsrc/http_custom.c 
esp82xx/fwsrc/mdns.c esp82xx/fwsrc/mfs.c user/custom_commands.c 
user/user_main.c -flto -Wl,--relax -Wl,--gc-sections -nostdlib 
-L/home/romanvg/esp8266/esp_nonos_sdk/lib 
-L/home/romanvg/esp8266/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/libgcc.a /home/romanvg/esp8266/esp_nonos_sdk/lib/libmain.a 
/home/romanvg/esp8266/esp_nonos_sdk/lib/liblwip.a 
/home/romanvg/esp8266/esp_nonos_sdk/lib/libssl.a
/home/romanvg/esp8266/esp_nonos_sdk/lib/libupgrade.a
/home/romanvg/esp8266/esp_nonos_sdk/lib/libnet80211.a
/home/romanvg/esp8266/esp_nonos_sdk/lib/libwpa.a 
/home/romanvg/esp8266/esp_nonos_sdk/lib/libphy.a 
/home/romanvg/esp8266/esp_nonos_sdk/lib/liblwip.a
/home/romanvg/esp8266/esp_nonos_sdk/lib/libcrypto.a 
/home/romanvg/esp8266/esp_nonos_sdk/lib/libpp.a 
/home/romanvg/esp8266/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/libgcc.a 
-T esp82xx/ld/linkerscript.ld 
-T /home/romanvg/esp8266/esp_nonos_sdk/ld/eagle.rom.addr.v6.ld 
-B/home/romanvg/esp8266/esp_nonos_sdk/lib  -o image.elf

So I guess you are suggesting that I should create a lmic.a and then add it as a section in the irom0 linkerscript section or tag most of the LMIC functions as ICACHE_FLASH_ATTR?

[cnlohr]

Correct!

[brainstorm]

Thanks @cnlohr! Bizarre, I've just done so and went down just a few bytes (20 bytes from my last attempt). By the message from the linker it seems to stick in iram1 still though:

~/dev/espressif/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld:
 image.elf section `.text' will not fit in region `iram1_0_seg'
~/dev/espressif/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/../../../../xtensa-lx106-elf/bin/ld:
 region `iram1_0_seg' overflowed by 5604 bytes
collect2: error: ld returned 1 exit status

Here are the sizes of the .o's and the resulting liblmic.a lib:

(py2) romanvg@romanvg:~/dev/esp8266/esp82XX-basic-LMIC$ ls -alh user/lmic/liblmic.a 
-rw-r--r-- 1 romanvg romanvg 64K Feb 15 16:51 user/lmic/liblmic.a
(py2) romanvg@romanvg:~/dev/esp8266/esp82XX-basic-LMIC$ ls -alh user/lmic/*.o 
-rw-r--r-- 1 romanvg romanvg 9.3K Feb 15 16:51 user/lmic/aes.o
-rw-r--r-- 1 romanvg romanvg 3.0K Feb 15 16:51 user/lmic/hal.o
-rw-r--r-- 1 romanvg romanvg  36K Feb 15 16:51 user/lmic/lmic.o
-rw-r--r-- 1 romanvg romanvg 3.7K Feb 15 16:51 user/lmic/oslmic.o
-rw-r--r-- 1 romanvg romanvg  12K Feb 15 16:51 user/lmic/radio.o

I added the above liblmic.a into linkerscript.ld's section of esp82xx:

  .irom0.text : ALIGN(4)
  {
    _irom0_text_start = ABSOLUTE(.);

    *libmbedtls.a:(.literal .text .literal.* .text.*)
    *liblwip.a:(.literal .text .literal.* .text.*)
    *liblmic.a:(.literal .text .literal.* .text.*)      /* LoRa LMIC library */

    *(.irom0.literal .irom.literal .irom.text.literal .irom0.text .irom.text)
    _irom0_text_end = ABSOLUTE(.);
  } >irom0_0_seg :irom0_0_phdr

This is how I compiled each .c's from the LMIC lib:

$(CC) -mlongcalls -Os -I/home/romanvg/esp8266/esp_nonos_sdk/include -Iesp82xx/include -I. -Iuser/lmic -Iesp82xx/fwsrc -Iuser -DICACHE_FLASH -DCFG_eu868 -DCFG_DEBUG -DCFG_sx1272_radio -DCFG_BARE_LORA_ESP82XX -nostdlib -L/home/romanvg/esp8266/esp_nonos_sdk/lib -B/home/romanvg/esp8266/esp_nonos_sdk/lib user/lmic/hal.c -c -o user/lmic/hal.o

Don't know why that lib is still sticking to iram, any clues? :/

[projectgus]

A couple of things that might help:

• Does your linker pass produce a .map file? Argument -Wl,-Map=something.map, or similar? The linker map is the best way we have to figure out what is going where in your linker output, and why.
• It's probably not related to this particular problem, but I would always recommend compiling with --ffunction-sections -fdata-sections and linking with --gc-sections. This means that unused functions or symbols in each object file won't be linked into the final binary if they're not used. Otherwise everything is linked at the object file level, which means that if there are 5 functions in one object file and you only use one of them, all five are still taking up space in your binary (including IRAM if they link to IRAM).
• Are you linking lmic with -llmic, or is it possible the .o files themselves are being passed to the linker? Because the linker script addition above will only work if the files are being pulled from the archive file ("library").

[brainstorm]

Thanks much Angus for the tips!

At least one of the LMIC symbols ends up in the .map file I'm generating via make debug, I pasted the full dump over pastebin... not sure what the $1409 at the end of the symbol means though:

3ffe84b8 000001ac b LMIC$1409

The above falls within the iram0 range according to the MEMORY section in the linkerscript:

dram0_0_seg : org = 0x3FFE8000, len = 0x14000

Then, afterwards I've introduced the (object size optimization?) flags you suggested:

LMIC_OPTS = -ffunction-sections -fdata-sections -Wl,--gc-sections

Compiling only the LMIC components like so:

$(CC) -mlongcalls -Os $(LMIC_OPTS) 
-I/home/romanvg/esp8266/esp_nonos_sdk/include -Iesp82xx/include -I. -Iuser/lmic 
-Iesp82xx/fwsrc -Iuser -DICACHE_FLASH -DCFG_eu868 
-DCFG_DEBUG -DCFG_sx1272_radio -DCFG_BARE_LORA_ESP82XX 
-nostdlib -L/home/romanvg/esp8266/esp_nonos_sdk/lib 
-B/home/romanvg/esp8266/esp_nonos_sdk/lib user/lmic/hal.c  -c - o user/lmic/hal.o

And this is the result with LMIC_OPTS on:

(py2) romanvg@romanvg:~/dev/esp8266/esp82XX-basic-LMIC$ ls -alh user/lmic/*.o
-rw-r--r-- 1 romanvg romanvg 9.7K Feb 16 17:37 user/lmic/aes.o
-rw-r--r-- 1 romanvg romanvg 4.7K Feb 16 17:37 user/lmic/hal.o
-rw-r--r-- 1 romanvg romanvg  60K Feb 16 17:37 user/lmic/lmic.o
-rw-r--r-- 1 romanvg romanvg 5.6K Feb 16 17:37 user/lmic/oslmic.o
-rw-r--r-- 1 romanvg romanvg  19K Feb 16 17:37 user/lmic/radio.o

And without them (undefined/empty LMIC_OPTS):

$ ls -alh user/lmic/*.o
-rw-r--r-- 1 romanvg romanvg 9.3K Feb 16 17:42 user/lmic/aes.o
-rw-r--r-- 1 romanvg romanvg 3.0K Feb 16 17:42 user/lmic/hal.o
-rw-r--r-- 1 romanvg romanvg  36K Feb 16 17:42 user/lmic/lmic.o
-rw-r--r-- 1 romanvg romanvg 3.7K Feb 16 17:42 user/lmic/oslmic.o
-rw-r--r-- 1 romanvg romanvg  12K Feb 16 17:42 user/lmic/radio.o

So I'm really confused now on what the compiler/linker actually does behind the scenes with those flags :-S

Last but not least, I'm making sure that no lmic-related .o files lie behind after creating the archive file:

    $(AR) r user/lmic/liblmic.a user/lmic/*.o
    $(RM) user/lmic/*.o

And then include the archive file along the rest of the esp_nonos_sdk (since if this all works, would like to include it as optional part of the Espressif SDK, of course). For this I just run make all from cnlohr's main.mf target:

(...)
-flto -Wl,--relax -Wl,--gc-sections -nostdlib -L/home/romanvg/esp8266/esp_nonos_sdk/lib 
-L/home/romanvg/esp8266/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/libgcc.a
/home/romanvg/esp8266/esp_nonos_sdk/lib/libmain.a 
/home/romanvg/esp8266/esp_nonos_sdk/lib/liblwip.a 
(...)
/home/romanvg/esp8266/esp_nonos_sdk/lib/libcrypto.a
/home/romanvg/esp8266/esp_nonos_sdk/lib/liblmic.a <------ LMIC archive file
/home/romanvg/esp8266/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.5/libgcc.a 
-T esp82xx/ld/linkerscript.ld 
-T /home/romanvg/esp8266/esp_nonos_sdk/ld/eagle.rom.addr.v6.ld 
-B /home/romanvg/esp8266/esp_nonos_sdk/lib  -o image.elf