search

Aircoookie / Espalexa

Alexa voice control for ESP8266/ESP32 (including brightness and color!)
MIT License
531 stars 135 forks source link

devices not found compiling foresp32 arduino core #41

Open robademar opened 5 years ago

robademar commented 5 years ago

I've updated esp32 arduino core and now it is correctly compiled. But as opposite to the esp8266, discovery doesn't find any device. I tried changing cpu frequency to 240Mhz, before at lower speed the board was continuously resetting, but no way to fiìnd the devices. I added debug and this is the log: rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff0018,len:4 load:0x3fff001c,len:1100 load:0x40078000,len:10088 load:0x40080400,len:6380 entry 0x400806a4 [D][WiFiGeneric.cpp:342] _eventCallback(): Event: 0 - WIFI_READY [D][WiFiGeneric.cpp:342] _eventCallback(): Event: 2 - STA_START

Connecting to WiFi Connecting.....[D][WiFiGeneric.cpp:342] _eventCallback(): Event: 4 - STA_CONNECTED [D][WiFiGeneric.cpp:342] _eventCallback(): Event: 7 - STA_GOT_IP [D][WiFiGeneric.cpp:385] _eventCallback(): STA IP: 192.168.1.6, MASK: 255.255.255.0, GW: 192.168.1.1 . Connected to Vodafone-roberto IP address: 192.168.1.6 Constructing device 1 Adding device 1 Constructing device 2 Adding device 2 Adding device 3 Espalexa Begin... MAXDEVICES 10 Done Got UDP! NOTIFY * HTTP/1.1 HOST: 239.255.255.250:1900 CACHE-CONTROL: max-age=100 LOCATION: http://192.168.1.250:49152/description.xml NT: urn:schemas-wifialliance-org:service:WFAWLANConfig:1 NTS: ssdp:alive SERVER: Linux/2.6.36, UPnP/1.0, Portable SDK for U

any suggestion? thanks

Originally posted by @robademar in https://github.com/Aircoookie/Espalexa/issues/40#issuecomment-458240790

Aircoookie commented 5 years ago

This is really strange. In the debug, I see no discovery requests, so they are not coming through somehow. I would have suspected your network, but in that case the ESP8266 shouldn't work either. I'll get back to you once I have an idea.

robademar commented 5 years ago

I used the same code for both esp8266 and 32 . this is the log of esp8266 test started from my browser esp log.txt

If you successfully tested the esp32 version I would think that there is something wrong in my arduino environment. For the board I used the last available version I downloaded after your suggestion . thanks for your time

eos1d3 commented 5 years ago

@Aircoookie

Hi, got some hints now.

  1. Example of EspalexaBasic.ino using ESP32, none can be discovered.
  2. Without debug, I tried so many times to discover, nothing can be found.
  3. Adding #define ESPALEXA_DEBUG before Espalexa.h, all three devices always be discovered.

I repeated these tests three times (three times with debug, three times without debug), each time with debug, it is easy to find all three devices. And on/off operations work too.

Did you comment out some useful codes?

PS: look like I found the bug.

in Espalexa.h, line 128 - 129, you need to put the two EA_DEBUGLN in a block. This fixes the discovery problem even without DEBUG option. Please confirm this.

void serveNotFound()
  {
    EA_DEBUGLN("Not-Found HTTP call:");
    #ifndef ESPALEXA_ASYNC
    EA_DEBUGLN("URI: " + server->uri());
    EA_DEBUGLN("Body: " + server->arg(0));
    if(!handleAlexaApiCall(server->uri(), server->arg(0)))
    #else
    { // <-- this one
        EA_DEBUGLN("URI: " + server->url());
        EA_DEBUGLN("Body: " + body);
    } // <-- and this one
    if(!handleAlexaApiCall(server))
    #endif
    server->send(404, "text/plain", "Not Found (espalexa-internal)");
  }

PPS:

Use alexa web page is reliable. Old Alexa app does not work. But the latest version work. Its GUI is completely changed.

robademar commented 5 years ago

my Espalexa.h was contains already the changes you suggested . This is:

//not found URI (only if internal webserver is used) void serveNotFound() { EA_DEBUGLN("Not-Found HTTP call:");

ifndef ESPALEXA_ASYNC

EA_DEBUGLN("URI: " + server->uri());
EA_DEBUGLN("Body: " + server->arg(0));
if(!handleAlexaApiCall(server->uri(), server->arg(0)))
#else
EA_DEBUGLN("URI: " + server->url());
EA_DEBUGLN("Body: " + body);
if(!handleAlexaApiCall(server))
#endif
  server->send(404, "text/plain", "Not Found (espalexa-internal)");

}

and I used the debug option in my code, but it doesn't work.

eos1d3 commented 5 years ago

Your version is still the original one. Where are the brackets?

And do not use old Alexa app to discover which never works for me. Use https://alexa.amazon.com or latest Alexa app, both work!

PS: There is NO need to set UPnP too. If still not work, change router if you have another. I saw people have problem with some router. Asus routers mostly OK.

robademar commented 5 years ago

sorry I've misread the suggestion. Now I changed but still not working I tried with web browser, ios app and alexa discovery. No message comes through. this is the log log after changes to Espalexa_h.docx at this point I guess it is something in my environment, but I just reloaded the esp32 board sw.

eos1d3 commented 5 years ago

@robademar I suddenly could not find any more with more 10 trials.

Then I switched my phone WIFI to 2.4HGz from 5GHz, it finds now!

I read similar problem from one using FauxmoESP. I am still testing...

Please try without DEBUG too..

robademar commented 5 years ago

debug removed, same result. My phone is on a 5 GHz network

eos1d3 commented 5 years ago

44 Read this. I think there is fundamental problem for ESP32 version. Even when it is found, it will freeze and not response after a period of time. Similar to what I am testing now, sometimes it works, sometimes it does not.

But earlier for discovery is reliable, but not now.

And no such problem for ESP82166.

Aircoookie commented 5 years ago

Putting the EA_DEBUGLN in brackets shouldn't really affect anything, non-debug-mode discovery worked for me without them, so I can't confirm this. I don't know why 5Ghz would be an issue (maybe it depends on the router, could be that some models don't forward broadcast packets to the 2.4G network).

I noticed a different issue though, sometimes the ESP32 doesn't connect to the network and you have to hit the reset button for it to connect. This happens both in debug and normal mode, as well as async and non-async. I believe it is an DHCP issue, but not sure on that one.

@eos1d3 once again, thank you for all your help :)

eos1d3 commented 5 years ago

For discovery issue, I would like to talk here. This is a more solid test results.

I suspect Amazon is changing their server right right now. I have the same latest Alexa Android app for 2 days, but I never see discovery have notifications on Android!!

Just around an hour ago, when I successfully discovered ESP-8266 devices, There were three notifications on my mobile phone: "New Light named Light x found. To control it, say "Alexa, turn off Light x". At the same time, Alexa will say by voice of similar message. And the Alexa app icon has a red 3 showing. All these were never seen before.

I am still seeing this notifications most of the time after discovery. But I did found at least once no notification at all but devices are found.

And for ESP8266, the discovery is a lot better now. Using Alexa app get 80% to find the devices. Using Alexa web page just gets around 50 - 60% to find.

I left home for around one hour. After that, I used my phone to control ESP8266 devices, they are still there and no error at all! This is so nice to see.

For ESP32, it can be found, but may be less than 10% chance for Alexa App. Almost never found when using Alexa web page. (But yesterday, it is almost as good as ESP8266)

But device controls will break networking of ESP32.

During my tests, UPnP is not affecting any operation. And I almost disable it during tests now. Actually I did not see any port is opened by UPnP even I enabled it.

eos1d3 commented 5 years ago

And for ESP32 log, this is before discover operation:

Connecting to WiFi
Connecting........
Connected to xxx
IP address: 192.168.2.134
Constructing device 1
Adding device 1
Constructing device 2
Adding device 2
Adding device 3
Espalexa Begin...
MAXDEVICES 10
Done
Got UDP!
M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 1
ST: urn:dial-multiscreen-org:service:dial:1

Got UDP!
M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 1
ST: urn:dial-multiscreen-org:service:dial:1

Got UDP!
M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 1
ST: urn:dial-multiscreen-org:service:dial:1

Got UDP!
NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
ETHERNET: 00.06.dc.81.d7.a5
CACHE-CONTROL: max-age=1800
LOCATION: http://169.254.178.125:2020/myiBoxUPnP/description.xml
NT: urn:schemas-upnp-org:device:SyabasSTB:1
NTS: ssdp:alive
SERVER: Linux/SMP86xx

Sometimes, I see no M-SEARCH * HTTP/1.1 section, sometimes I see 1 to 3 times as shown above.

What is the normal log?

eos1d3 commented 5 years ago

@Aircoookie

5G WIFI issue

I read that from FauxmoESP, someone said this problem for FauxmoESP. But for my test with your library, my mobile phone is on 5G (or my PC with Ethernet), ESP8266 on 2.4GHz, it just works! xD

Putting the EA_DEBUGLN in brackets shouldn't really affect anything

I will test again later.

sometimes the ESP32 doesn't connect to the network

Both ESP8266 and ESP32 have this issue. But I always make sure they are connected before discovery. So there is never a connection issue for my tests.

Your library is nice. And the Alexa app can control device on/off/dim by just touching the buttons on mobile phone (I do not know if these exist before). This is excellent! I like this so much.

Aircoookie commented 5 years ago

That's interesting, I've never got any notifications about discovered devices before either. Yes, I can also confirm that ESP8266 seems to be a lot more reliable than ESP32 currently. Did you also try #define ESPALEXA_ASYNC? The async version seems to work a bit better for me.

The M-SEARCH packets with ST: urn:dial-multiscreen-org:service:dial:1 are unrelated and sent by other services (router, phones, media devices) . Alexa discovery M-SEARCH packets either have ST upnp:rootdevice or device:Basic:1.

Thank you! Yes, control in the app is very handy and it even works if your phone is not in the home network!

eos1d3 commented 5 years ago

If so, Amazon is really changing something. But the same Alexa app did not show any notifications yesterday until today afternoon!

I am completely new to your library. I never know there is ESPALEXA_ASYNC. Will try later.

Yesterday ESP32 discovery worked very nice. I never change any code or network. And it so hard to discover today! So strange!

Can Amazon server is updating some servers o that we may be connected to some new and old server at the same time? I see no point it can't work.

I will keep testing later. Thank you too.

eos1d3 commented 5 years ago

@Aircoookie Amazon is definitely playing with their server side right now!!

I used "https://alexa.amazon.com/" to discover devices with my Mac. And suddenly my phone got three discovery notifications and Echo dot said the same things!

So these notifications are not from new version of Alexa app. It is from Amazon server side!

eos1d3 commented 5 years ago

@Aircoookie Just tested with ESPALEXA_ASYNC, all problems are the same. Around 20% can discover devices. Controlling devices will disconnect ESP32 too.

What timeout settings for related network requests? My ESP calls Amazon server from Asia, I expect network may take longer time. Just a guess as I am seeing the discovering so unpredictable.

Aircoookie commented 5 years ago

I believe the communication is Phone <-> Amazon Server <-> Echo <-> ESP device, so the notifications would make sense. However, the discovery itself shouldn't be related to the server at all, it is only between the Echo and the ESP. First, the Echo sends broadcast SSDP discovery requests. The ESP answers those, then the Echo will "follow up" and request device details which the ESP will provide. At no point the ESP will connect to any Amazon server by itself (it could be that the Echo communicates with Amazon servers during the process though to add the new device to the database). Still your argument with the server could be valid, I live in Germany (Ireland Alexa server), so that could make a difference, since I'm not having any discovery issues (but I am still affected by #44, so I'll try to investigate some more about that ASAP. The fact that /espalexa stops working means that the ESP is no longer connected to the network, so there must be a cause for that, most likely a bug in an Espalexa method)

eos1d3 commented 5 years ago

discovery itself shouldn't be related to the server at all

Agree with this.

But the problem is that I seldom see the discovery request from debug log. As shown below, when I start the ESP32 module, most of the time I see this, and after I start to discover, no new message is added. If I can see the discover request, there will be a lot messages and all devices must be able to be discovered.

Today ESP32 is still hard to be discovered.

In HK, Alexa is not supported officially by Amazon. I do not know if this causes problem.

ESP8266 is a lot better. But I still see sometimes its devices are not found. Usually if they are not found, the next discovery must find them. The success rate is more than 80%, unlike ESP32 which is almost none. I can't find once today.

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
Falling back to built-in command interpreter.
OK
>ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00

mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:928
ho 0 tail 12 room 4
load:0x40078000,len:9280
load:0x40080400,len:5860
entry 0x40080698

Connecting to WiFi
Connecting........
Connected to xxx
IP address: 192.168.2.134
Constructing device 1
Adding device 1
Constructing device 2
Adding device 2
Adding device 3
Espalexa Begin...
MAXDEVICES 10
Done
Got UDP!
NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
ETHERNET: 00.06.dc.81.d7.a5
CACHE-CONTROL: max-age=1800
LOCATION: http://169.254.115.38:2020/myiBoxUPnP/description.xml
NT: urn:schemas-upnp-org:device:SyabasSTB:1
NTS: ssdp:alive
SERVER: Linux/SMP86xx,

(at here, no more message are shown even I keep discovering...)
eos1d3 commented 5 years ago

A little update with ESP8266.

Now using Alexa web page to discover get 100% success. Test with more than 5 times. And each time there is one notification for each device.

And the sequent for each devices of appearance is a little different. Before all three devices will appear together at the same time. Now, each will appear separately. It is around some seconds between each.

So ESP8266 work flawlessly. While there is no luck for ESP32.

Is port 80 being used? This says only port 80 works for Echo Dot Gen 3 which is the one I am using. https://bitbucket.org/xoseperez/fauxmoesp

eos1d3 commented 5 years ago

@Aircoookie I found a way to let ESP32 devices to be discovered now. With 100% success rate! Tested with ESPALEXA_ASYNC and also without ESPALEXA_ASYNC!

This is what I am doing:

  1. Use Alexa App, let Discovery button ready to press, but do not press it. Let your finger above it!
  2. Reset ESP32
  3. When you see WIFI is connected, with "MAXDEVICES 10 & Done", press the Discovery button immediately!!
  4. All devices are there with notifications!

Tested more than 5 times without a failure! Before this, I get no success at all.

But there is still a strange problem. Alexa App sometime will display no device is found, but all three devices will appear with notifications a few second later. And sometimes, it showed 2 or 1 devices were found, but all three will be there.

PS: I will be away for Chinese New Year and will be back later.

Aircoookie commented 5 years ago

@eos1d3 thanks for all the testing! The high ESP8266 success rate leads me to believe that this has nothing to do with your location. Yes, Espalexa uses port 80 (the hue api is http based after all).

When you see WIFI is connected, with "MAXDEVICES 10 & Done", press the Discovery button immediately!!

The fact that this works makes me believe that this "no discovery" issue and #44 might be the same and the ESP32 just loses the connection really early (possibly when receiving the first SSDP request that is not by Alexa). I think so because the debug logs you posted from when it didn't work all show only a single received UDP packet, but always show it (either an unrelated M-SEARCH or a NOTIFY). I will investigate for memory leaks or similar faults which may occur after the first packet was received and get back to you if I have some results.

But there is still a strange problem. Alexa App sometime will display no device is found, but all three devices will appear with notifications a few second later. And sometimes, it showed 2 or 1 devices were found, but all three will be there.

This is common for me as well. I believe that sometimes, even if the devices are correctly added to the database, the app somehow doesn't always receive feedback about the discovery in time.

Have a happy and lucky new year!

eos1d3 commented 5 years ago

The fact that this works makes me believe that this "no discovery" issue and #44 might be the same and the ESP32 just loses the connection really early (possibly when receiving the first SSDP request that is not by Alexa).

I do not know if connection dropped and reconnected, but each time after discovery, ESP32 is always connected. Only Alexa app control operations for on/off/dim will break connection.

And you are right, whenever there is "NOTIFY" in log, it won't be discovered.

See these:

Before discovery:
Free Heap: 272540
Uptime: 252792

After discovery:
Free Heap: 272532
Uptime: 276655

I think you may find the bug very soon. Happy new year too!

robademar commented 5 years ago

BRAVOOO! First tests are ok tomorrow I will do more

Thanks Gio

eos1d3 commented 5 years ago

Confirmed version 2.3.3 fixed discovery bug for ESP32. Notifications are always there for each device. Great work!