Build sketches without bootloader

[lucysrausch]

Basic Infos

Is it possible to compile sketches without eboot bootloader?

Description

Hey,

I would like to use the rboot bootloader (https://github.com/raburton/rboot) to have multiple ROMs on my ESP written using Arduino. For this, I need the compiled .elf, but without the eboot bootloader so I can create my own image using esptool2. Is there any flag or anything I can comment out to accomplish this?

Thanks in advance, Niklas

[davisonja]

You need to edit the build recipes in platform.text, probably specifically

recipe.objcopy.hex.pattern

Though I suspect it won't support any of the OTA updatery functionality. You may also need to update some addresses of things (I don't know how large the rboot binary is, if it's more than a single page (4k) things will get more complicated and you'll need to edit the linker files.

[lucysrausch]

Thanks for the fast answer, the platform.txt was exactly what I was looking for!

First, I made a copy of the nodemcu board config and edited the corresponding linker script by increasing the irom0_0_seg address from 0x40201010 to 0x40202010 (rboot is a bit bigger than eboot).

I changed ## Combine gc-sections, archives, and objects recipe.c.combine.pattern="{compiler.path}{compiler.c.elf.cmd}" {compiler.c.elf.flags} {compiler.c.elf.extra_flags} -o "{build.path}/{build.project_name}.elfraw" [...]

so I get the .elfraw without the bootloader, that seems to work.

Then I use esptool2 to create my own binary:

esptool2 -bin -debug -boot2 -1024 -dio /tmp/arduino_build_236811/sketch.ino.elfraw ~/rboot_test/rom0.bin .irom0.text .text .data .rodata

I upload everything using esptool: esptool.py --port /dev/ttyUSB0 -b 921600 write_flash --flash_mode dio --flash_size 1MB 0x0 rboot.bin 0x02000 rom0.bin 0x82000 rom1.bin

The rboot loader seems to work fine, however, the actual rom crashes:

Serial dump: `rBoot v1.4.2 - richardaburton@gmail.com Flash Size: 8 Mbit Flash Mode: DIO Flash Speed: 40 MHz

Everything up to date. Booting rom 0.

Fatal exception (3): epc1=0x4000438f, epc2=0x00000000, epc3=0x00000000, excvaddr=0x40202010, depc=0x00000000 `

So, any idea why this happens?

Best regards, Niklas

[davisonja]

What's at 0x4000438f? :) A map file from the linker is useful for finding things like that.

Did you follow the rboot 'Linking user code' section? A quick glance through rboot's code suggests there shouldn't be any major barriers to doing what you're trying to.

[lucysrausch]

Thanks for the hint!

I generated the map using xtensa-lx106-elf-nm

40003a14 A Uart_Init 40004a4c A SPIWrite 40004b1c A SPIRead 400043c8 A SPI_read_status 4000443c A SPI_write_enable 40004400 A SPI_write_status

EDIT: I tried other sketches like the basic blink sketch, the esp always crashes at 0x4000438f :/

For now I use the addresses from the provided rboot example. One question: Why do I need to specify the SPI mode, size and clock for creating the binary? I never had problems using other clock rates or bigger flash sizes. Actually, the board I'm developing with has 8MB of flash, even though it's not supported by esptool I can program it without any problems setting the flash size to 1MB, 2MB or 4MB. Might this be a problem?

[davisonja]

It's crashing in the middle of a SPIRead call, which I suspect means it hasn't left rboot, which would mean the sketch is irrelevant. Do you have a binary from something that's not the Arduino system that works?

[devyte]

I believe there's a PR of nodemcu firmware (the one with lua integrated) with a working rboot. I seem to recall an effort to expose the rboot api in an attempt to implement OTA updates (the firmware doesn't have OTA capability yet). You could try taking a look at how that was done.

On Apr 3, 2017 10:21 PM, "Julian Davison" notifications@github.com wrote:

It's crashing in the middle of a SPIRead call, which I suspect means it hasn't left rboot, which would mean the sketch is irrelevant. Do you have a binary from something that's not the Arduino system that works?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/esp8266/Arduino/issues/3111#issuecomment-291362648, or mute the thread https://github.com/notifications/unsubscribe-auth/AQC6BrlQQJOyZIGSj7DfQq8-tez286GPks5rsZstgaJpZM4MyG91 .

[lucysrausch]

So, I tried a few more things, for example I'm able to execute my "standalone" roms using the eboot bootloader, having the rom at 0x1000. So the problem must be with rboot.

I had a close look on the actual bootloading thing, both eboot and rboot are using "jx a3" for setting the instruction pointer. I noticed this line in eboot that is missing in rboot:

register uint32_t sp asm("a1") = 0x3ffffff0;

Can someone explain me what it does?

Best regards, Niklas

[davisonja]

I expect (given the surrounding lines) that it's initialising the 'stack pointer' or 'sp' to be the end of RAM

[TheOriginalMrWolf]

Are you using the right flash mode (DIO)? Most are QIO.

[devyte]

@NiklasFauth did you get this to work? To be honest, I would love to move to rboot, it is better known and has better features vs. eboot (at least at first glance). In addition, I fully support your effort to have multiple images flashed. The OTA process would certainly be safer, because you would always write the "other image", and then just change the boot address to point to it. Then, if for some reason the boot of the new image failed, you could fall back to the "old image". As I explained in my previous comment, I understand that the Nodemcu Lua project uses rboot successfully. I believe that the current approach with eboot is different. During OTA, there is a fixed "emtpy" area at which the new image is flashed. Then, during boot, the new image is copied over the old normal image. The consecuence of that is, if there is a power outage during the copy process, the ESP could get soft-bricked.

[devyte]

Reference: #905

[davisonja]

That's issue #905 :)

Should we be looking at switching to rboot? It's been a while since I've looked at it, but does it some how detect that 'the boot of the new image failed'? Aside from checks that the image is correct (such as a checksum) I'm not sure how the bootloader would know...

[davisonja]

For reference, the solution for #905 is to only mark the update as complete when the copy process has completed successfully; until it has, each startup results in the copy being tried again.

[devyte]

@davisonja I don't know the specifics, maybe it's possible to do something similar with eboot as well. Even so, I like rboot better. It's open source and well known. The way I understand the Nodemcu logic is that you don't mark the update as complete until very late in the boot process. I think in our case it would be just before calling setup. The point of the logic is to not just make sure that the update finished successfully, but to also make sure that the new image actually boots, at least up until a certain point. If the new image fails to boot, then you mark the update as failed, and roll back to the previous image. Example: say that the new image was built wrongly (wrong layout, a code change in main, whatever). You update the ESP, the process succeeds. But then it won't boot anymore => the ESP is soft-bricked.

The Nodemcu logic is something along these lines: The flash has 2 image slots, call them 0 and 1. Boot from current image (say slot 0). Update slot 1 with broken image, process succeeds. Boot to image in slot1. Boot process fails (i.e.: doesn't reach the point where the image is marked "good"), fall back to reboot from image in slot 0. => no soft brick.

I think there are other details as well, such as if there is an immediate reboot with boot reason crash, then the new image is also discarded and the previous image is booted. I'm not sure of the specifics, it's been a long while since that discussion.

[davisonja]

@devyte Okay, that makes sense. As long as the new image knows to mark itself as being okay, that would work happily.

I'm happy to look at rboot as an alternative. eboot is very simple and, I think, fits nicely with the relatively simple approach of arduino (not overly complex, but functional), but it might be nice to have the option of using rboot for those who would like more complex behaviour.

[igrr]

Integrating rboot might be easier if you work within the context of makeEspArduino or platform.io. I can imagine that shoehorning another link step into Arduino IDE build process can be done, but would be quite hacky (you somehow need to tell Arduino to compile bootloader files, but not link them into the sketch). Alternative would be to put rboot elf file into the repository, as it is done now with eboot, but one would probably loose a fair bit of rboot's configurability that way. I would prefer to keep the option of using eboot in Arduino, but i'm not religious about this. If rboot can be integrated nicely, I would not mind.

[davisonja]

I think eboot should be the default. It's simple, it works, and it does what the vast majority of novice users want - which is not to say we can't improve it :)

rboot offers a wider range of functionality that, in my opinion, is well beyond a novice users use-case, but baked in, or a documented process to use, rboot would be useful enough it's worth pursuing.

On Fri, Sep 8, 2017 at 8:53 PM, Ivan Grokhotkov notifications@github.com wrote:

Integrating rboot might be easier if you work within the context of makeEspArduino or platform.io. I can imagine that shoehorning another link step into Arduino IDE build process can be done, but would be quite hacky (you somehow need to tell Arduino to compile bootloader files, but not link them into the sketch). Alternative would be to put rboot elf file into the repository, as it is done now with eboot, but one would probably loose a fair bit of rboot's configurability that way. I would prefer to keep the option of using eboot in Arduino, but i'm not religious about this. If rboot can be integrated nicely, I would not mind.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/esp8266/Arduino/issues/3111#issuecomment-328044461, or mute the thread https://github.com/notifications/unsubscribe-auth/AAN_A6iOM9A6K1Ubh07iBxXGwYjixFpCks5sgQCUgaJpZM4MyG91 .

[milkpirate]

That's issue #905 :)

Should we be looking at switching to rboot? It's been a while since I've looked at it, but does it some how detect that 'the boot of the new image failed'? Aside from checks that the image is correct (such as a checksum) I'm not sure how the bootloader would know...

Hey,

this thread is old but I would like to share my thought about rboot.

At work we are dealing with RAUC. Which is for Linux the equivalent of rboot for whatever you want to call the flash content of an ESP.

Anyway I quite like RAUC and it has nice features like failovers, bundle signitures, compatability strings and so on. It also has a mark-good command which tags the running image ...well: as good. But defining conditions to trigger this command is up to the architects/users.

Whats does a successful boot mean? That LEDs are blinking? All drivers were loaded successfully? That there's a wifi connection? That there's a internet connection? That the correct credentails for whatever are present?

This is not easy to define or check, especially not for the bootloader. rboot does its best to verify you have a working slot (to use RAUC terminology) - which is more than eboot does - and boots it, what then happens might be: hang, watchdog, reboot, backup slot - which, to my understanding, is not possible with eboot. If a slot fails, its up to programmer of that slot, not the bootloader. RAUC (and rboot - if configured correctly) try to maintain the device reachable as well as possible and this is a huge advantage over eboot.

With rboot it is possible to realize nearly all what RAUC can do - especially fallback on broken updates (experiences it quite often that an OTA bricked my ESPs and I had to reprogram them by serial).

So I would relly like to see rboot beeing available to Arduino.

[5chufti]

I would say - for this usecase - a boot was successfull if setup() was started. I opt for a precompiled rboot replacing eboot so as to not change the general behaviour. So pros can "precompile" their bells-and-whistles rboot while for the general arduino user additional "nice-to-have" rboot features could gradually be implemented ...