Getting `[E][WiFiGeneric.cpp:658] hostByName(): DNS Failed` randomly

[GeorgeFlorian]

Hello !

I am using a HTTP Client inside loop to check for the body of a URL. It worked fine until it didn't. Today the ESP crashed and restarted. After the restart it connected to the WiFi using SSID and Password, so using DHCP. But it behaved like it did not connect.

• The web-pages that I have on the ESPAsyncWebServer didn't work.
• And I repeatedly got:

  [E][WiFiGeneric.cpp:658] hostByName(): DNS Failed
  [HTTP] GET... failed, error: connection refused

This is a part of the code that runs in void loop():

if (WiFi.status() == WL_CONNECTED && SPIFFS.exists("/configFile.txt")) {  //Check the current connection status 
      if(URL.length() > 15) {
        HTTPClient http;
        http.begin(URL); //Specify the URL
        int httpCode = http.GET(); //Make the request  
        if (httpCode > 0) { //Check for the returning code
          logOutput((String)"[HTTP] GET... code: " + httpCode);
          if(httpCode == HTTP_CODE_OK) {
              String payload = http.getString();
              logOutput(payload);
              signPlace = getPlaces(payload);
              oldSignPlace = signPlace;
            }
        } else {
            logOutput((String)"[HTTP] GET... failed, error: " + http.errorToString(httpCode).c_str());
          }  
        http.end(); //Free the resources
        } else {
            logOutput("URL Link is invalid ! Please enter another URL");
        }
    } // if (WiFi.status() == WL_CONNECTED)

Manually restarting the ESP, by pressing on the EN button, did fix the issue, however I still get this randomly. The only fix is to restart again !

[atanisoft]

can you confirm the value of URL is valid at all times? It may make sense to add a log output to print the URL being attempted.

[GeorgeFlorian]

can you confirm the value of URL is valid at all times? It may make sense to add a log output to print the URL being attempted.

URL starts as follows: String URL = "INVALID_URL"; It's length is 11.

First condition comes from HTML: <input id="text-input" type="url" inputmode="url" placeholder="Enter URL" name="getURL" value="" title = "URL"> I won't be able to input anything else aside a URL.

Second condition comes from the web-server: if(URL != "INVALID_URL" && URL != NULL && URL.length() > 15)

Third condition is the one in void loop() : if(URL.length() > 15) {}

URL = "INVALID_URL" won't be able to arrive at http.begin(URL); The HTTP Client won't even be initialized until I have a valid URL.

I have tried with the same URL over and over again. I sometimes get the error and sometimes it works perfectly.

It may make sense to add a log output to print the URL being attempted.

I don't follow.

My logOutput is a method I made to push Strings into a buffer so that I can print over a %Placeholder% that's inside a HTML page.

void logOutput(String string1) {
    circle.push(string1);   
    Serial.println(string1);    
}

[atanisoft]

What I meant was can you add a printf (or Serial.println in your case likely) to dump the URL value just before you call http.begin(URL); I suspect that you are ending up with a partial value in the URL string that is causing the DNS failure.

[GeorgeFlorian]

What I meant was can you add a printf (or Serial.println in your case likely) to dump the URL value just before you call http.begin(URL); I suspect that you are ending up with a partial value in the URL string that is causing the DNS failure.

I've added:

        Serial.println((String)"DEBUG: URL = " + URL);
        http.begin(URL); //Specify the URL

For now it seems that it works.

[GeorgeFlorian]

@atanisoft So it happened again. Now I can confirm that the URL that reaches http.begin(URL); is correct and that I still I get the error after an ESP Crash.

[E][WiFiGeneric.cpp:658] hostByName(): DNS Failed
[HTTP] GET... failed, error: connection refused

My workaround is to have a counter each time if (httpCode > 0) is not true. When the counter hits a certain value I perform a restart:

void loop() {
if (httpCode > 0) {
\*
*\
} else {
            logOutput((String)"[HTTP] GET... failed, error: " + http.errorToString(httpCode).c_str());
            httpFailCounter++;
          }
\*
*\
  if(httpFailCounter > 5) {
    delay(100);
    ESP.restart();
  }
}

What do you think causes [E][WiFiGeneric.cpp:658] hostByName(): DNS Failed ?

[atanisoft]

That means it failed to resolve the hostname. There are a number of reasons for this and usually a retry should fix it.

[atanisoft]

You should also check the return value from http.begin() as it should return false when dns fails.

[GeorgeFlorian]

You should also check the return value from http.begin() as it should return false when dns fails.

Ok. I will further investigate using this: logOutput(http.begin(URL) ? "DNS Resolved" : "DNS Fail");

My concern is that the DNS fails only after a crash. I get a [HTTP] GET... failed, error: connection refused from time to time, without [E][WiFiGeneric.cpp:658] hostByName(): DNS Failed, but this seems normal and it works fine next loop.

[GeorgeFlorian]

@atanisoft Sorry to bother you, but I get a strange behavior when disconnecting the ESP and then plugging it back. Using this code:

    if (WiFi.status() == WL_CONNECTED && SPIFFS.exists("/configFile.txt")) {  //Check the current connection status
      logOutput((String)"IP ADDRESS: " + IP_copy.toString());
      if(URL.length() > 15) {
        HTTPClient http;
        Serial.println((String)"DEBUG: URL = " + URL);
        logOutput(http.begin(URL) ? "DNS Resolved" : "DNS Failed");
        int httpCode = http.GET(); //Make the request  
        // Serial.println("int httpCode = http.GET();");
        if (httpCode > 0) { //Check for the returning code
          // Serial.println("if (httpCode > 0)");
          logOutput((String)"[HTTP] GET... code: " + httpCode);
          if(httpCode == HTTP_CODE_OK) {
            // Serial.println("if(httpCode == HTTP_CODE_OK)");
            httpFailCounter = 0;
            String payload = http.getString();
            logOutput(payload);
            signPlace = getPlaces(payload);
            oldSignPlace = signPlace;
          }
        } else {
            logOutput((String)"[HTTP] GET... failed, error: " + http.errorToString(httpCode).c_str());
            httpFailCounter++;
          }  
        http.end(); //Free the resources
        } else {
            logOutput("URL Link is invalid ! Please enter another URL");
        }
    } // if (WiFi.status() == WL_CONNECTED)

Serial.println((String)"DEBUG: URL = " + URL); returns the correct URL. logOutput(http.begin(URL) ? "DNS Resolved" : "DNS Failed"); returns DNS Resolved

And then comes [HTTP] GET... failed, error: connection refused. The only thing that fixes this is pressing the EN button to reset the ESP or , what I am currently doing, using a counter to count the number of errors and on 4 straight errors the ESP restarts using ESP.restart().

Do you have any idea why is this happening ?

[atanisoft]

Unfortunately I'm not sure why it is doing that

[maxpromer]

I have same this bug. and now i use counter to check dns fail if over 6 times i use ESP.restart() to restart ESP. after restart ESP can connect to server without this bug.

i think fix this bug without restart ESP is better.

[justoke]

I can confirm this issue, as I have been investigating the same problem for a couple of weeks now. These are my observations:

Devices and Configuration

ESP32 Dev Module PubSubClient - MQTT library MQTT Broker - cloudmqtt.com Arduino platform on Platform IO

Error Logged

[E][WiFiGeneric.cpp:652] hostByName(): DNS Failed for m20.cloudmqtt.com

Behaviour

Once the ESP32 has connected to WiFi and MQTT client is connected, the code runs tasks that read sensors and other typical microcontroller tasks. I usually have the MQTT web interface open so I can see messages going in and out of the ESP32. Now the dns failure can happen randomly. And sometimes hours or even just a few minutes after the device has been powered up or reset.

WiFi Stack Game Over

However, once it has failed it will not recover. I had to implement a similar regime as @maxpromer and do an ESP.restart() after a set number of failures. I tried reconnecting WiFi client but this issue seems to break the WiFi stack and only a reset will get it working.

Host is online, DNS resolution is fine

When it is in this DNS failed state, I can nslookup the domain from my PC and the MQTT web interface is working fine. This issue affects the fundamental operation of an IOT device - it must be able to maintain its connection to the WiFi AP so that it can access resources such as MQTT brokers, http APIs and so forth. I now make the assumption that the internet is down when I implement my code, as the WiFi stack is not reliable enough to maintain a connection or recover from outages and failed DNS resolution.

Proposed Change

It seems it would be safer to assume that once DNS has been resolved, that the resolved IP address could be cached. The IP address for mqtt.com, for example, has not changed in over a year. The DNS failure could raise a callback, that would pass the last successfully resolved IP address and the developer could then attempt to connection using the LastSuccessfulIPAddress, instead of the host name. If this fails to connect then, a few retries of the same patterns and we could assume with some certainty that the host may well be offline or unavailable.

ESP8266 By Comparison

I have a couple of ESP8266 running the same MQTT client and they been online for 18 months now and I have never had to reset them, not even once. They have survived internet outages, MQTT outages, simply I assume because the WiFi stack recovers from these issues and keeps a healthy connection.

[macedolfm]

Hi, may be my experience can shed some light at the DNS mistery as I myself was also struck by the infamous:

"[E][WiFiGeneric.cpp:652] hostByName(): DNS Failed...blá, blá ,blá"

My code was running rock solid for months in a ESP32 Heltec LoRa board using WiFi+MQTTS+TLS=> Mosquitto Server. After hours of debugging I found out the culprit, a invalid IP string being converted by IPAddress.formString(). The boolean result was not checked in code and the bad IPAddress result was used in a WiFi.softAPConfig() setup. After that line the WiFi stack started to pop all sort of random errors including DNS and SSL failures. Fixed the invalid string and a we are back on heaven again. Hope it might help.

[a-c-sreedhar-reddy]

If you are facing this issue can you once try restarting WIFI router and confirm please..

[justoke]

If you are facing this issue can you once try restarting WIFI router and confirm please..

It is always the ESP32 that has to be restarted - router restart does not help, as the WiFi never recovers the connection after the DNS error message.

[justoke]

Hi, may be my experience can shed some light at the DNS mistery as I myself was also struck by the infamous:

"[E][WiFiGeneric.cpp:652] hostByName(): DNS Failed...blá, blá ,blá"

My code was running rock solid for months in a ESP32 Heltec LoRa board using WiFi+MQTTS+TLS=> Mosquitto Server. After hours of debugging I found out the culprit, a invalid IP string being converted by IPAddress.formString(). The boolean result was not checked in code and the bad IPAddress result was used in a WiFi.softAPConfig() setup. After that line the WiFi stack started to pop all sort of random errors including DNS and SSL failures. Fixed the invalid string and a we are back on heaven again. Hope it might help.

Thank you for the suggestion. I'll look through the code - always worth checking with things like this. I'm still bothered by the fact that the device cannot recover from this situation. For an IOT device, this is a disaster as it will not recover and be offline. If remotely deployed, then this is even more of an issue.

[gicamois2019]

Hello. I am having the same problem. By some reason, the call http.begin("web address"); gives the infamous error and causes the ESP32 to crash. I have implemented the counter for resetting the device, but it does solve the problem in all cases. I still have my device frozen from time to time (once a day maybe, device connecting to the AP and sending the data once per minute). It is true that now this happens more rarely. If I think that I have to place tens of devices in remote locations, I get a bit concerned.

[lewisxhe]

This is really bad. I check the server for online requests every ten seconds. After a few minutes of continuous requests, the server disconnects and then connects again. It will also return DNS Failed for xxxxx. Very badly, every few Need to restart once in minutes

[a-c-sreedhar-reddy]

This is really bad. I check the server for online requests every ten seconds. After a few minutes of continuous requests, the server disconnects and then connects again. It will also return DNS Failed for xxxxx. Very badly, every few Need to restart once in minutes

This is not a fix. But what I do is reconnect.

[justoke]

Anyone from the Espressif team that could at least give an opinion on this?

With almost 1000 issues on this project, prioritizing is obviously a challenge. However, I would say that any issue that concerns WiFi is worthy of some attention. It is after all the basis of IOT capability.

[lewisxhe]

I can confirm this issue, as I have been investigating the same problem for a couple of weeks now. These are my observations:

Devices and Configuration

ESP32 Dev Module PubSubClient - MQTT library MQTT Broker - cloudmqtt.com Arduino platform on Platform IO

Error Logged

[E][WiFiGeneric.cpp:652] hostByName(): DNS Failed for m20.cloudmqtt.com

Behaviour

Once the ESP32 has connected to WiFi and MQTT client is connected, the code runs tasks that read sensors and other typical microcontroller tasks. I usually have the MQTT web interface open so I can see messages going in and out of the ESP32. Now the dns failure can happen randomly. And sometimes hours or even just a few minutes after the device has been powered up or reset.

WiFi Stack Game Over

However, once it has failed it will not recover. I had to implement a similar regime as @maxpromer and do an ESP.restart() after a set number of failures. I tried reconnecting WiFi client but this issue seems to break the WiFi stack and only a reset will get it working.

Host is online, DNS resolution is fine

When it is in this DNS failed state, I can nslookup the domain from my PC and the MQTT web interface is working fine. This issue affects the fundamental operation of an IOT device - it must be able to maintain its connection to the WiFi AP so that it can access resources such as MQTT brokers, http APIs and so forth. I now make the assumption that the internet is down when I implement my code, as the WiFi stack is not reliable enough to maintain a connection or recover from outages and failed DNS resolution.

Proposed Change

It seems it would be safer to assume that once DNS has been resolved, that the resolved IP address could be cached. The IP address for mqtt.com, for example, has not changed in over a year. The DNS failure could raise a callback, that would pass the last successfully resolved IP address and the developer could then attempt to connection using the LastSuccessfulIPAddress, instead of the host name. If this fails to connect then, a few retries of the same patterns and we could assume with some certainty that the host may well be offline or unavailable.

ESP8266 By Comparison

I have a couple of ESP8266 running the same MQTT client and they been online for 18 months now and I have never had to reset them, not even once. They have survived internet outages, MQTT outages, simply I assume because the WiFi stack recovers from these issues and keeps a healthy connection.

I saw what you said compared to 8266. I put the code running on on 8266. It works fine, although it will also be offline from the server, but it works fine again, instead of needing to restart.

[BillCilla]

I am also getting this problem on ESP32 when the code works perfectly on ESP8266. I get: [E][WiFiGeneric.cpp:652] hostByName(): DNS Failed for api.timezonedb.com Although my host and request work ok if I just edit them into the ESP32 WiFiClient.ino example. Its when I add all the rest of my code that the problem starts (too much code to include here). I'm using: Arduino IDE 1.8.9, Espressif ESP32 library version 1.02. Included libraries are: , , which replace , , for the ESP8266 version.

[lewisxhe]

I am connecting to the same server in the esp-idf framework, it works fine, so this is definitely a problem with arduino-esp32, I have to run the code in esp-idf because it is stable

[atanisoft]

this is definitely a problem with arduino-esp32, I have to run the code in esp-idf because it is stable

IDF has a few bug fixes which have not made it over to the arduino-esp32 side likely.

[Daemach]

I'm seeing the same problem. Once it starts failing it will not recover and must be restarted. Debug output looks like this:

[D][HTTPClient.cpp:265] beginInternal(): host: l.thcguard.com port: 80 url: /xyz.php?sn=CC50E3AA5718&fv=41&cv=37&tm=2019.08.20.09.06.01&r=100&t=74&t2=0&h=53&h2=0&c=430&c2=0&i=454&vc=127&s=0&a=0&b=0.00&ll=0&li=0&lf=0&lv=0&lx=65535&lc=0&lr=0&lg=0&lb=0&ta=0.0&ha=0.0&ca=0.0&ia=0.0&la=0.0&vx=0.0
[E][WiFiGeneric.cpp:652] hostByName(): DNS Failed for l.thcguard.com
[D][HTTPClient.cpp:995] connect(): failed connect to l.thcguard.com:80
[W][HTTPClient.cpp:1287] returnError(): error(-1): connection refused
09:06:05        198092  logData attempt failed with error: connection refused (-1)
[D][HTTPClient.cpp:369] disconnect(): tcp is closed

There is no corruption in the URL and anything that uses HTTPClient, including HTTPUpdate, fails once this starts occurring. The only solution is to restart.

[D][HTTPClient.cpp:265] beginInternal(): host: f.thcguard.com port: 80 url: /abc.php?sn=CC50E3AA5718&fv=41
[E][WiFiGeneric.cpp:652] hostByName(): DNS Failed for f.thcguard.com
[D][HTTPClient.cpp:995] connect(): failed connect to f.thcguard.com:80
[W][HTTPClient.cpp:1287] returnError(): error(-1): connection refused
[E][HTTPUpdate.cpp:217] handleUpdate(): HTTP error: connection refused

[D][HTTPClient.cpp:369] disconnect(): tcp is closed

09:24:01        197960  FWUpdate failed:

High water mark for the main loop is 5048 and freeHeap is 200K+. Oddly, if I compile with debug/verbose output it runs much longer before failing, but does start failing eventually. And it fails the same way in both 1.02 and 1.03RC.

[afulloa]

Hello to all, First excuseme for my English Language. I have de same problem. I got de error at compiling at date 16/8/2019, aproximatly. I have recompiling at 12/08/2019 and it was run fine. But recompiling the new revision of the aplication at 16/08/2019 I get the same error as your have. I make a test, compiling the example from ESP32 in HTTPClinet/examples/BasicHttpClient.ino and I have :+1:

[HTTP] begin... [W][HTTPClient.cpp:236] beginInternal(): unexpected protocol: https, expected http [D][HTTPClient.cpp:265] beginInternal(): host: www.example.org port: 443 url: /index.html [HTTP] GET... [E][WiFiGeneric.cpp:652] hostByName(): DNS Failed for www.example.org [E][WiFiClientSecure.cpp:132] connect(): start_ssl_client: -1 [D][HTTPClient.cpp:995] connect(): failed connect to www.example.org:443 [W][HTTPClient.cpp:1287] returnError(): error(-1): connection refused [HTTP] GET... failed, error: connection refused [D][HTTPClient.cpp:369] disconnect(): tcp is closed

I suspect thar can be a driver problem in the drivers or IDE.
In both I use de R 1.02.

Thanks

[eklass]

Hello guys,

The ESP32 turn me into trouble when I try to connect to myHost with the WifiClient. I get the error message:

[WiFiGeneric.cpp:652] hostByName(): DNS Failed for [myHost]

But the point is, if I connect the WifiClient to the host: data.sparkfun.com with the port 80 everything works well. MyHost is a webserver from webgo (if it is relevant). From my computer it is possible to reach myHost, but only with my esp32 it does not work.

I also tried to set the wifi-settings manually, but the same result: sparkfun works, myHost not.

IP address: 192.XXX.YYYY.ZZZZ
ESP Mac Address: AA:BB:CC:DD:EE:FF
Subnet Mask: 255.XXX.YYY.ZZZ
Gateway IP: 192.XXX.YYY.ZZZ
DNS: 8.8.8.8

Do you have any suggestions what I can do?

Thank you in advance!

[stale[bot]]

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions.

[asier70]

I had similar problem using VPN connection. I've probably found bug in library source code. Look at file: Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi\src\WiFiGeneric.cpp

/**
 * DNS callback
 * @param name
 * @param ipaddr
 * @param callback_arg
 */
static void wifi_dns_found_callback(const char *name, const ip_addr_t *ipaddr, void *callback_arg)
{
    if(ipaddr) {
        (*reinterpret_cast<IPAddress*>(callback_arg)) = ipaddr->u_addr.ip4.addr;
    }
    xEventGroupSetBits(_network_event_group, WIFI_DNS_DONE_BIT);
}

/**
 * Resolve the given hostname to an IP address.
 * @param aHostname     Name to be resolved
 * @param aResult       IPAddress structure to store the returned IP address
 * @return 1 if aIPAddrString was successfully converted to an IP address,
 *          else error code
 */
int WiFiGenericClass::hostByName(const char* aHostname, IPAddress& aResult)
{
    ip_addr_t addr;
    aResult = static_cast<uint32_t>(0);
    waitStatusBits(WIFI_DNS_IDLE_BIT, 5000);
    clearStatusBits(WIFI_DNS_IDLE_BIT);
    err_t err = dns_gethostbyname(aHostname, &addr, &wifi_dns_found_callback, &aResult);
    if(err == ERR_OK && addr.u_addr.ip4.addr) {
        aResult = addr.u_addr.ip4.addr;
    } else if(err == ERR_INPROGRESS) {
        waitStatusBits(WIFI_DNS_DONE_BIT, 4000);
        clearStatusBits(WIFI_DNS_DONE_BIT);
    }
    setStatusBits(WIFI_DNS_IDLE_BIT);
    if((uint32_t)aResult == 0){
        log_e("DNS Failed for %s", aHostname);
    }
    return (uint32_t)aResult != 0;
} 

If the answer from DNS server is very slow and sometimes it takes over 4000ms, WIFI_DNS_DONE_BIT is set by callback after this timeout and remains set. In the next DNS ask, function waitStatusBits() exits immediatelly because this bit is set, aResult is zero so function returns 0 (error) and then http.GET() returns error -1 (connection refused). I think the best way is to insert additional code at the beginning of hostByName(): clearStatusBits(WIFI_DNS_DONE_BIT);

[justoke]

Please could this issue be re-opened - it does not appear to be resolved.

[asier70]

I also postulate that this issue should be re-opened. I think that the two library function cited by me should be reviewed by the author or someone else and corrected. Probably there is second serious issue in them. When DNS resolving lasts more than 4000ms, function hostByName() exits, but callback() will be called and variable aResult will be updated. Unfortunatelly all functions that use WiFiGenericClass::hostByName() e.g.:

int WiFiClient::connect(const char *host, uint16_t port, int32_t timeout)
{
    IPAddress srv((uint32_t)0);
    if(!WiFiGenericClass::hostByName(host, srv)){
        return 0;
    }
    return connect(srv, port, timeout);
} 

use the parameter srv (internally aResult) as variable declared on stack (or heap?). After timeout this memory area used now by another function may be changed by callback. That may generate unknown troubles that needs esp32 to be restarted..

[ajyta]

Hi, after long debugging-journey I also found out that WiFiClientSecure failed at connect with UNKNOWN ERROR (-1).

Unfortunatelly there are many reasons, why "start_ssl_client", which is called by WiFiClientSecure::connect returns -1, and in my case it was because WiFiGenericClass::hostByName failed.

Test case: Wifi connection droped, after it was established again, a connect to WiFiClientSecure was done <- WHICH FAILED due to hostByName

asier70's proposal did the fix:

I think the best way is to insert additional code at the beginning of hostByName(): 'clearStatusBits(WIFI_DNS_DONE_BIT);`

The only thing, which I'm still wondering, that Stack Watermark is stressed a lot during my tests. I have not investigated in this, but the following prick my ears:

use the parameter srv (internally aResult) as variable declared on stack (or heap?). After timeout this memory area used now by another function may be changed by callback. That may generate unknown troubles that needs esp32 to be restarted..

Could you explain a little more, please?

And yes, you are right, this issue should be reopened....

BR, PM

[asier70]

Sorry for my late answer. Look at WiFiClient.cpp file in library and this function:

int WiFiClient::connect(const char *host, uint16_t port, int32_t timeout)
{
    IPAddress srv((uint32_t)0);
    if(!WiFiGenericClass::hostByName(host, srv)){
        return 0;
    }
    return connect(srv, port, timeout);
}

At file ssl_client.cpp it looks similiar:

int start_ssl_client(sslclient_context *ssl_client, const char *host, uint32_t port, int timeout, const char *rootCABuff, const char *cli_cert, const char *cli_key, const char *pskIdent, const char *psKey)
{
    ...
    IPAddress srv((uint32_t)0);
    if(!WiFiGenericClass::hostByName(host, srv)){
        return -1;
    } 
    ...
}

Object srv is local and placed on stack (temporary object of IPAddress class). pointer to this object is passed to hostByName() function and then it is used by DNS callback function. Function hostByName() waits only 4 seconds if DNS operation is in progress. If DNS operation lasts more time it returns with error but internal DNS operation still works and at the end callback function is called. Callback function still has pointer to object srv that doesn't exists at this moment, so it writes resolved IP address to this memory area. Probably this stack memory area is already used by other function so this can have unknown consequences to application work :( Is anybody know how to reopen this issue?

[asier70]

This thread is closed so I decided to open new issue #3722 to show that this problem exists and should be solved.

[ajyta]

Thanks for opening a new issue.