OTA on HTTP server - ESPhttpUpdate.update return HTTP_UPDATE_FAILED Error (-1)

[odilonafonso]

Basic Infos

Hardware

Hardware: NodeMCU 0.9 - (ESP-12 module) Core Version: 2.3.0-rc2

Description

I'm trying to upgrade the sketch on an HTTP server, over the internet (OTA). The first question I have is, where is the OTA code, responsible for replacing the old with the new sketch? In the example shown there is no reference to the OTA libraries.

The second: It is possible to keep the two form the update in the same sketch - the Arduino IDE with OTA libraries and a Http Server?

Settings in IDE

Module: NodeMCU 0.9 - (ESP-12 module) Flash Size: 4MB/3MB? CPU Frequency: 80Mhz Upload Using: OTA

Sketch

WebUpdater from ESP8266HTTPUpdateServer library

[odilonafonso]

The procedure to update the sketch for my network wifi worked, but not fully in accordance with the documentation. In the documentation, item 5 says:

To Obtain the file navigate to directory used by Arduino IDE to store results of compilation. You can check the path to this file in compilation log shown in IDE debug window the marked below.

With this image: https://github.com/esp8266/Arduino/blob/master/doc/ota_updates/ota-web-path-to-binary.png

But I did not get it in my IDE debug window; I had to examine more carefully the log and found that reference to where the new compiled sketch was recorded:

"/tmp/buildfc496aee2b83585546553dcc21e915b4.tmp/WebUpdater.ino.bin"

And the upload message:

Uploading 257680 bytes from to flash at 0x00000000

And not the message shown in the example, pointing to where the sketch was compiled, such as:

Uploading 257680 bytes from /tmp/buildfc496aee2b83585546553dcc21e915b4.tmp/WebUpdater.ino.bin to flash at 0x00000000

I had no trouble in pointing out that the file upload procedure HTTPUpdateServer. It worked very well, but not like this complicated name and where he was.

Now I would like to know where the make file compilation is, so I can add a copy command of the newly compiled sketch to a directory with more friendly name, so can facilitate future updates.

Can anyone tell where is located the make file to build? And if I can interact with it, to achieve this result I want?

[odilonafonso]

Now I am trying use the ESPhttpUpdate class, to update from Httpserver.

But I am receiving:

HTTP_UPDATE_FAILED Error (-1): HTTP error: connection refused

any suggestions of what might be happening?

The sketch that I am using is the httpUpdate with minor modifications:

#define SERVER "http://odilon.local/" // /web/robuino/bin/"
#define SKETCH_BIN "httpUpdate.ino.bin"
void loop() {
    // wait for WiFi connection
    if((WiFiMulti.run() == WL_CONNECTED)) {

        String binFile = SERVER;
        binFile += SKETCH_BIN;

        t_httpUpdate_return ret = ESPhttpUpdate.update(binFile);
        //t_httpUpdate_return  ret = ESPhttpUpdate.update("https://server/file.bin");

        switch(ret) {
            case HTTP_UPDATE_FAILED:
                USE_SERIAL.printf("HTTP_UPDATE_FAILED Error (%d): %s\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
                USE_SERIAL.println(binFile);
                break;

            case HTTP_UPDATE_NO_UPDATES:
                USE_SERIAL.println("HTTP_UPDATE_NO_UPDATES");
                break;

            case HTTP_UPDATE_OK:
                USE_SERIAL.println("HTTP_UPDATE_OK");
                break;
        }
    }
}

[odilonafonso]

May be some configuration on my Apache ??

tried this new sketch:

#define LOCAL_SERVER "http://odilon.local"
#define REMOTE_SERVER "http://robuino.com.br"
#define SKETCH_BIN "/file.bin"
void loop() {
    // wait for WiFi connection
    if((WiFiMulti.run() == WL_CONNECTED)) {

        t_httpUpdate_return ret = ESPhttpUpdate.update(REMOTE_SERVER, 80, SKETCH_BIN);

        switch(ret) {
            case HTTP_UPDATE_FAILED:
                USE_SERIAL.printf("HTTP_UPDATE_FAILED Error (%d): %s\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
                USE_SERIAL.print(REMOTE_SERVER);USE_SERIAL.println(SKETCH_BIN);
                break;

            case HTTP_UPDATE_NO_UPDATES:
                USE_SERIAL.println("HTTP_UPDATE_NO_UPDATES");
                break;

            case HTTP_UPDATE_OK:
                USE_SERIAL.println("HTTP_UPDATE_OK");
                break;
        }
    }
}

unsuccessfully

[krzychb]

Hi @odilonafonso,

Your server seems to operate correctly. I have tried it by replacing in example sketch httpUpdate.ino the following line:

t_httpUpdate_return ret = ESPhttpUpdate.update("http://server/file.bin"); with t_httpUpdate_return ret = ESPhttpUpdate.update("http://robuino.com.br/file.bin");

The file.bin file loads on my ESP without any issues.

If not done already I suggest you follow excellent tutorial #78 First IOT Appstore: Remote Update of your ESP8266 from Webserver by @SensorsIot that explains in great details how OTA HTTP Server method works.

[krzychb]

Re:

The procedure to update the sketch for my network wifi worked, but not fully in accordance with the documentation.

@odilonafonso, could you attach the screenshot that you see instead of what shown in documentation?

[odilonafonso]

Hello @krzychb,

Thanks again for help.

Ok, I change the way to call ESPhttpUpdate, now is working. I change the bin in the webserver, now the name is:

httpUpdate.ino.bin

But, I don't know why:

t_httpUpdate_return ret = ESPhttpUpdate.update("http://robuino.com.br/httpUpdate.ino.bin");

works

and

t_httpUpdate_return ret = ESPhttpUpdate.update("http://robuino.com.br", 80, "/httpUpdate.ino.bin");

do not works.

Another question is:

Why the code never goes to the switch block when ESPhttpUpdate.update() succesfull ? This switch has no default; when ret == 0 it goes to HTTP_UPDATE_FAILED case. It is correct ?

Where can I see the values of the constants: HTTP_UPDATE_FAILED, HTTP_UPDATE_NO_UPDATE and HTTP_UPDATE_OK ?

I am quite satisfied, in general is working very well !!! I need to find some other details of how the update works, but it will happen to the medium in which I delve into and do more tests.

I'll send you a screen shot copy where the location of the newly compiled sketch is not informed. It is below.

Again, thank you for your precious help!

[odilonafonso]

Hi @krzychb, Hear the screen shot from Arduino IDE after the compilation process. I hope that the image is well readable.

[krzychb]

Hello @odilonafonso,

Why the code never goes to the switch block when ESPhttpUpdate.update() succesfull ?

When ESPhttpUpdate.update() is successful, then the module is soft restarted to run newly loaded code instead.

Thank you for the screenshot! I think that under File > Preferences you have checked out “upload” under “Show verbose output during:” Therefore the screen looks different than in documentation (it contains extensive upload log).

[odilonafonso]

I was unable to update the ESP by a web server with:

ESPhttpUpdate.update (server, port, uri);

But with the method:

ESPhttpUpdate.update (url);

It's working very well.

And I found out how to make the switch after the firmware upgrade pack to run - there is a method to determine (or not) to reset the ESP:

void ESPhttpUpdate.rebootOnUpdate (bool reboot);

And through this method it is possible to tell how it should be done the update.

I made some small changes in httpUpdate.ino sample and tested it on my local server; worked perfectly.

Grateful for the help @krzychb

Sketch

/**
 * httpUpdate.ino
 *
 *  Created on: 27.11.2015
 *
 */

#include <Arduino.h>

#include <ESP8266WiFiMulti.h>
#include <ESP8266httpUpdate.h>

#define USE_SERIAL Serial

#define SSID "ssid"
#define PASSWORD "passw"

ESP8266WiFiMulti WiFiMulti;

#define DEBUG_ESP_PORT Serial
#define DEBUG ESP_HTTP_UPDATE

#define THIS_VERSION " V0.2 "
#define BOOT_AFTER_UPDATE false

#define LED_STATUS 0
void blinkLed(int led){
   for (int i=0; i< 5; i++){
      digitalWrite(led, HIGH);
      delay(111);
      digitalWrite(led, LOW);
      delay(111);
   }
   delay(500);
}

void setup() {
   pinMode(LED_STATUS, OUTPUT);
   digitalWrite(LED_STATUS, LOW);
   blinkLed(LED_STATUS);

   USE_SERIAL.begin(115200);
   USE_SERIAL.setDebugOutput(true);

   USE_SERIAL.println();
   USE_SERIAL.println();
   USE_SERIAL.println();

   for(uint8_t t = 4; t > 0; t--) {
     USE_SERIAL.printf("[SETUP] WAIT %d...\n", t);
     USE_SERIAL.flush();
     delay(1000);
   }

   USE_SERIAL.print("\nhttpUpdate");
   USE_SERIAL.print(THIS_VERSION);
   USE_SERIAL.println("start...");

   WiFiMulti.addAP(SSID, PASSWORD);

}

#define LOCAL_SERVER "http://odilon.local"
#define REMOTE_SERVER "http://robuino.com.br"
#define SKETCH_BIN "/httpUpdate.ino.bin"
int updateCount = 1;
void loop() {
   blinkLed(LED_STATUS);
   // wait for WiFi connection
   if((WiFiMulti.run() == WL_CONNECTED)) {

      USE_SERIAL.print("\nhttpUpdate");
      USE_SERIAL.print(THIS_VERSION);
      USE_SERIAL.print("loop...");
      USE_SERIAL.println(updateCount++);

      String url = REMOTE_SERVER; //LOCAL_SERVER
      url += SKETCH_BIN;
      USE_SERIAL.print("url: ");USE_SERIAL.println(url);
      USE_SERIAL.print("REMOTE_SERVER: ");USE_SERIAL.println(REMOTE_SERVER);
      USE_SERIAL.print("SKETCH_BIN: ");USE_SERIAL.println(SKETCH_BIN);
//      t_httpUpdate_return ret = ESPhttpUpdate.update(REMOTE_SERVER, 80, SKETCH_BIN);
      //seta ESP8266.rebootOnUpdate - false - para não rebootar (teste)
      ESPhttpUpdate.rebootOnUpdate(BOOT_AFTER_UPDATE);
      t_httpUpdate_return ret = ESPhttpUpdate.update(url);
      USE_SERIAL.print("ret ");USE_SERIAL.println(ret);

      switch(ret) {
         case HTTP_UPDATE_FAILED:
             USE_SERIAL.printf("HTTP_UPDATE_FAILED Error (%d): %s\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
             break;

         case HTTP_UPDATE_NO_UPDATES:
             USE_SERIAL.println("HTTP_UPDATE_NO_UPDATES");
             break;

         case HTTP_UPDATE_OK:
             USE_SERIAL.println("HTTP_UPDATE_OK");
             break;

         default:
             USE_SERIAL.print("Undefined HTTP_UPDATE Code: ");USE_SERIAL.println(ret);

        }
    }
}

[odilonafonso]

I would like to save the file ".bin" compiled. The makefile used by the Arduino IDE writes this ".bin" to a temporary directory; to close the IDE this directory is deleted and the next time I open the IDE, it will be created a new one.

Does anyone know where to find this Makefile?

Hi @krzychb, I checked the option “upload” under “Show verbose output during:” But keep getting the same message ... must be something else.

By the way, now the httpUpdate.ino.bin file on the "robuino.com.br" server is not booting after update...

I used the:

ESPhttpUpdate.rebootOnUpdate(false);

method and this way, after update, the active sketch keep running.

If you update your ESP with this bin, you will see:

httpUpdate V0.2(no boot after update) start...

httpUpdate V0.2(no boot after update) loop...1

messages on your serial monitor. And next time:

httpUpdate V0.2(no boot after update) loop...2

and so on

It is very stable; I left this sketch running while doing other things and soon reached 380 updates ...

[krzychb]

@odilonafonso,

I would like to save the file ".bin" compiled.

You can save this file to your sketch folder by pressing Ctrl+Alt+S or going to Sketch > Export compiled Binary

Does anyone know where to find this Makefile?

Recipes for compilation, linking, etc are in platform.txt file.

Sometime ago I have prepared a short description how to add a new menu item to Arduino IDE. This involved modification of platform.txt file. Using this as a general idea you can prepare your own menu to copy the bin file to the specific folder you need. Note that the menu item "Upload Using" has been removed from Arduino IDE.

I checked the option “upload” under “Show verbose output during:”

This option should be unchecked like below to show only short upload log:

Krzysztof

[odilonafonso]

Hi @krzychb ,

You can save this file to your sketch folder by pressing Ctrl+Alt+S or going to Sketch > Export compiled Binary

Thanks ! This will help me a lot, until I automate the copy in the makefile - (platform.txt?)

Recipes for compilation, linking, etc are in platform.txt file.

Ok, thank you - I give a look to this file. But I don't know how to change some variables. Where are these defined variables: build.project_name, build.variant ??

 ## Save hex
 recipe.output.tmp_file={build.project_name}.bin
 recipe.output.save_file={build.project_name}.{build.variant}.bin

Sometime ago I have prepared a short description how to add a new menu item to Arduino IDE. This involved modification of platform.txt file. Using this as a general idea you can prepare your own menu to copy the bin file to the specific folder you need.

Useful this article !!! Well done ! I will give a good study of it.

Note that the menu item "Upload Using" has been removed from Arduino IDE.

My Arduino IDE shows "Upload using". What version do you use ? I use 1.6.10

This option should be unchecked like below to show only short upload log:

I tried all options, none of them has the name of the .bin file generated; this option shows:

Sketch uses 324,563 bytes (31%) of program storage space. Maximum is 1,044,464 bytes. Global variables use 40,980 bytes (50%) of dynamic memory, leaving 40,940 bytes for local variables. Maximum is 81,920 bytes. Uploading 328704 bytes from to flash at 0x00000000 .................................................................................................................................................................................................................................................................................................................................

No problem, now I know where the Arduino writes the .bin and the options you offered me I will find a way to put it where I want.

[krzychb]

@odilonafonso,

Ok, thank you - I give a look to this file. But I don't know how to change some variables. Where are these defined variables: build.project_name, build.variant ??

build.project_name -> this is the sketch name build.variant -> is mapped by the board you select under Tools > Board:, see https://github.com/esp8266/Arduino/blob/master/boards.txt#L28 For instance, if you select "Generic ESP8266 Module" then the build.variant will be generic, for "NodeMCU 0.9 (ESP-12 Module)" and "NodeMCU 1.0 (ESP-12E Module)" it will be nodemcu

My Arduino IDE shows "Upload using". What version do you use ? I use 1.6.10

I should have said "removed from boards.txt file", instead of "removed from Arduino IDE" - sorry. If you upgrade to stable esp8266 / Arduino core 2.3.0 you will not see it.

[odilonafonso]

@krzychb,

I think I need to work on {build.path] to get what I want. This variable is undefined and thus the compiler is generating a temporary directory in "/ tmp"

[Vamshi253]

Hi Odilonafonso,

Thanks for sharing your code.

At this moment I am engaged in developing the ESP8266 module which measures temperature and humidity.

Concerning Over The Air (OTA) update , I need to do it using HTTP Server and is based on continuous automated deployment of firmware on ESP8266.

But I cannot be able to find much resources in detailed on the internet.

Could you help me regarding this?

[odilonafonso]

Hello @Vamshi253,

It is a pleasure to share knowledge. I do not know how to help you, I'm a beginner too. I also have plans to develop an application to be accessed continuously and after checking a few different ways to do it, I opted for the use of MQTT. The firmware update via the Internet proved to be very efficient and practical, the performance obtained with the MQTT is very good, using websockets and while I have used public brokers, for testing. The results were very good.

The free brokers who tested satisfactorily are:

Eclipse - iot.eclipse.org HiveMw - http://www.hivemq.com/ Mosquitto - test.mosquitto.org

Of course, I installed MQTT in my notebook, is where I get the best performance to be dedicated. But all of the above have proved very good!

Unfortunately I do not have things very well yet documented, I do not dare to publish my experiences the way they are, but so forward a little more to things and deepen my knowledge, it will be a pleasure to disclose - even in return - this fantastic community both contributed to my learning.

At this point I'm documenting and making accessible three applications in principle for me and my friends:

A plant watering, with soil moisture sensor and drive a relay which is installed a solenoid or a water pump; A feeder for pets, consisting of a server feed and water; A trigger for gates and electronic doors.

And in my house, I have done a series of experiments with temperature readers and humidity (DHT11, DHT22), gas sensors LPG, distance, water flow, etc .. and a machine for making beer (yeahh !!!)

I do not know if that's what you wanted to know, maybe if you explain a little more goals I can help you more.

Good luck !

[Vamshi253]

Hello Odilonafonso,

Thank you for sharing your knowledge.

What I want is the implementation of OTA update using http server (web server). Specifically, the client (ESP8266 device) periodically checks for updates on the server through internet by itself and if any new version of firmware is available , it automatically updates by itself.

For this, we have two sides: 1) Client side and 2) Server side.

At client side, program is developed in arduino environment (OTA functionality).

Considering server side, a php app needs to be developed such that it provides the new firmware to the client based on MAC address or version number.

I have installed Apache server along with PHP support.

From this on, I cannot be able to move forward on how to setup and configure web server along with php app. I need a step by step guide which elaborates the configuration of web server along with php app to enable OTA functionality.

Could you help me regarding this?

[odilonafonso]

Hello @Vamshi253

I understood what you want to do.

Unfortunately right now I am very involved with other activities, otherwise it would be a pleasure to help you implement the way you want.

I do not think you will be very difficult to follow this guidance:

https://github.com/esp8266/Arduino/blob/master/doc/ota_updates/readme.md#advanced-updater

It is very similar to how you want to do and is very well explained what to do, I had no problems in implementing a simple upgrade, the advanced party did not have time to implement, but is deceptively simple.

I am running the update controlled form of firmware, run by a MQTT message - that is, the opposite way to that suggested by the above documentation.

The topic on which the ESP8266 receives several requests: soil moisture sensor, trigger relay, lamp light, open gate, etc .. etc ... also serves to communicate the availability of a new firmware release.

When I want to modify the firmware of my ESP, sending a message to the topic in which it is subscribed and ask him to proceed to the update. For example:

{ 'Action':'update', 'bin':'http: //myserver.com/bin/myesprobotv2.bin' }

When the ESP receives this message, it runs the same code as the tips given by our friend @krzychb hear:

https://github.com/esp8266/Arduino/issues/2482#issuecomment-244792933

I suggest that before you get busy with the advanced version, be sure to run a simple update.

[Vamshi253]

Hi @Odilonafonso,

Thank you very much for the information.

I followed your guidance and I got an idea of how to execute things.

But I did not get how to setup the web server and how to establish the protocol between web server and my esp through PHP script.

If possible can you please elaborate the steps?

[svh1985]

If you specify the port and the file as a separate parameters, you should not put http:// before the server address.

Like so:

//Works
ESPhttpUpdate.update("192.168.1.32", 80, "/WebUpdaterV2.bin");

//Doesn't work
ESPhttpUpdate.update("http://192.168.1.32", 80, "/WebUpdaterV2.bin");