search

maragelis / ParadoxRs232toMqtt

esp8266, serial bus to mqtt for Paradox alarm systems
MIT License
80 stars 22 forks source link

Wemos D1 Mini Pro - frequent reconnects #72

Open qntris opened 4 years ago

qntris commented 4 years ago

Hi,

I am using a Wemos D1 Mini Pro and a buck converter connected to the Gnd and + serial pins of the MG5050 alarm system. I have properly soldered the pins and checked the connections several times, confirmed that the system is working (I could read/arm/disarm using MQTT). However after ~20-30 minutes the Wemos D1 Mini Pro is disconnected (for no obvious reason) and does not come back online. I have resoldered the connections precisely, tested and connected back to the alarm system. Same thing happened again after a few minutes.

Now I have connected the power to the 3v pin (stepped down the voltage with the buck converter) and testing it for a few minutes but I still see disconnects - Socket error on client 84:F3:EB:DB:5B:06, disconnecting. (at least with quick reconnects this time).

Should I be targeting a faulty Wemos module? I have a NodeMCU board that I have already programmed and is waiting in the shelf. If those disconnects continue, I will be trying it instead of the Wemos board.

I have used the following libraries and settings, as it would not compile with some of the lib versions from the Readme: https://github.com/maragelis/ParadoxRs232toMqtt/issues/59#issuecomment-599254638

Using Arduino IDE v1.8.12 Install this from "Tools" -> "Board: "some board" -> "Boards Manager":

ESP8266 Community v2.6.3 Install these from "Tools" -> "Manage Libraries":

WiFiManager by tzapu v0.15.0 PubSubClient by Nick O'Leary 2.6.0 Arduino JSON lib version 5.13.5 (does not currently work with version 6.x) Files to be changed:

Open webpage.ino: Change "String page = FPSTR(HTTP_HEAD);" to "String page = FPSTR(HTTP_HEADER);" Change "page += FPSTR(HTTP_HEAD_END);" to "page += FPSTR(HTTP_HEADER_END);"

These are the Mosquitto logs: 1587933143: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user'). 1587933339: Socket error on client 84:F3:EB:DB:5B:06, disconnecting. 1587933360: New connection from 192.168.0.170 on port 1883. 1587933360: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user'). 1587933405: Socket error on client 84:F3:EB:DB:5B:06, disconnecting. 1587933405: New connection from 192.168.0.170 on port 1883. 1587933405: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user'). 1587933466: Socket error on client 84:F3:EB:DB:5B:06, disconnecting. 1587933487: New connection from 192.168.0.170 on port 1883. [INFO] found MQTT-user on Home Assistant 1587933489: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user'). 1587933669: Socket error on client 84:F3:EB:DB:5B:06, disconnecting. 1587933679: New connection from 192.168.0.170 on port 1883. 1587933679: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user'). 1587933770: Socket error on client 84:F3:EB:DB:5B:06, disconnecting. 1587933780: New connection from 192.168.0.170 on port 1883. 1587933780: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user'). 1587933835: Saving in-memory database to /data/mosquitto.db. 1587933885: Socket error on client 84:F3:EB:DB:5B:06, disconnecting. 1587933898: New connection from 192.168.0.170 on port 1883. [INFO] found MQTT-user on Home Assistant 1587933899: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user'). 1587933944: Socket error on client 84:F3:EB:DB:5B:06, disconnecting. 1587933967: New connection from 192.168.0.170 on port 1883. 1587933967: New client connected from 192.168.0.170 as 84:F3:EB:DB:5B:06 (p2, c1, k15, u'MQTT-user').

Angel0ffDeath commented 4 years ago

@qntris You can upgrade esp8266 core to 2.7.1 (last release) - tested. However try to compile with WiFiManager by tzapu v0.12.0 as it is stated in readme file and then report again the results

qntris commented 4 years ago

Hi @Angel0ffDeath, I have tried with: Arduino 1.8.12

The sketch won't compile for D1 Mini Pro: ` In file included from /Users/User/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.1/tools/sdk/libc/xtensa-lx106-elf/include/sys/stdio.h:6:0, from /Users/User/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.1/tools/sdk/libc/xtensa-lx106-elf/include/stdio.h:63, from /Users/User/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.1/cores/esp8266/Arduino.h:32, from sketch/ParadoxAlarmSystemOTA.ino.cpp:1: /Users/User/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.1/tools/sdk/libc/xtensa-lx106-elf/include/sys/pgmspace.h:25:130: error: 'const char HTTP_HEAD []' redeclared as different kind of symbol

define PROGMEM attribute((section( "\".irom.text." FILE "." STRINGIZE(LINE) "." STRINGIZE(COUNTER) "\"")))

                                                                                                                              ^

/Users/User/Documents/Arduino/libraries/WiFiManager/WiFiManager.h:25:24: note: in expansion of macro 'PROGMEM' const char HTTP_HEAD[] PROGMEM = "<!DOCTYPE html><html lang=\"en\"><meta name=\"viewport\" content=\"width=device-width, initial-scale=1, user-scalable=no\"/>{v}"; ^ In file included from /Users/User/Downloads/ParadoxRs232toMqtt-20190218/ParadoxAlarmSystem/ParadoxAlarmSystemOTA/ParadoxAlarmSystemOTA.ino:6:0: /Users/User/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.1/libraries/ESP8266WebServer/src/ESP8266WebServer.h:34:39: error: previous declaration of 'HTTPMethod HTTP_HEAD' enum HTTPMethod { HTTP_ANY, HTTP_GET, HTTP_HEAD, HTTP_POST, HTTP_PUT, HTTP_PATCH, HTTP_DELETE, HTTP_OPTIONS }; ^ exit status 1 Error compiling for board LOLIN(WEMOS) D1 mini Pro. `

qntris commented 4 years ago

Hi @Angel0ffDeath , I don't see any attachment and the post seems a bit off.

qntris commented 4 years ago

Can you zip it or upload somewhere and provide a link?

As for the reconnects, I am actually quite puzzled, the device works flawlessly:

My main router is TP-Link Archer C7 and the secondary one is TP-Link RE305 range extender, set up as Access Point, DHCP turned off, hooked up to the main router via cable. The main router and the RE305 are far away from each other, the beacon interval and the DITM interval is the lowest possible. The SSIDs and the passwords of the 2.4ghz and 5ghz networks are the same. Tried changing them as I suspected that the Wemos might be switching them, but that did not help either. I checked all the settings on my main router and on the RE305 and I honestly don't have any idea, why it would not be reconnecting when hooked up to the RE305 or when hooked up to the main router and being pinged from the RE305.... The signal strength is excellent in both rooms, the interference is very small. Other ESP8266 based devices (smart sockets) work without any issues in both rooms...

Maybe @maragelis can help here as I am left without any clues on what might be causing the strange reconnect issue when connected to my router and not pinged from the AP.

qntris commented 4 years ago

@Angel0ffDeath , can you delete your posts as they are spamming the topic - GitHub changes the content as it seems

Angel0ffDeath commented 4 years ago

Done... Sorry... Was reading my gmail and answering from there... Didn't know it is posted here also

Angel0ffDeath commented 4 years ago

@qntris - did you receive the file?

qntris commented 4 years ago

Yeap, will test it first thing tomorrow (it’s late on my end) and report back. I however doubt that this is the problem, as the board is quite steady on my AP and seems to be disturbed when on the main router...

Angel0ffDeath commented 4 years ago

just try, then disturb...

Angel0ffDeath commented 4 years ago

here is also late. Will wait your feedback

qntris commented 4 years ago

@Angel0ffDeath it compiled, but the result is the same - disconnects every couple minutes image Any ideas?

maragelis commented 4 years ago

What serial connections are you using. Software or hardware? . Rx/TX pins or the 5,6 pins (can’t remember exactly). Try using Hardware RX TX pins without enabling debug this the most stable way.

qntris commented 4 years ago

Hi @maragelis I use TX and RX, debug mode is per default settings - haven’t touched it. I moved the board to the other room, connected to the AP - very few disconnects (2 missed pings for 2 hours. However I see the following error in HomeAssistant’s Mosquitto’s log:

1591378630: Client 84:F3:EB:DB:5B:06 has exceeded timeout, disconnecting.

No other connection attempts after that. When pinging the board from a laptop connected to the WiFi, it is much more stable. My main board (NodeMCU) is connected to the main router and I am constantly pinging it from an old laptop that is connected to the AP in the other room (via WiFi) - no disconnects for 20-30 straight hours. I would run a trace but I am not sure how to do it - don’t have serial adapter (if needed).

Angel0ffDeath commented 4 years ago

@qntris - In addition to what @maragelis says I would like to add that in GENERAL WiFi works perfect. Your problems (as you describe them) are from the WiFi and not from Paradox communication. I am also with MG5050 and no problems here.

Nevertheless, first try what maragelis says and then additionally you can try the following: Check the signal strength. Check your WiFi router configuration - probably you would like to increase leasing time or to bound your alarm panel with certain IP. Also please adjust your DC-DC (Buck converter) to 5V.

Try to move your Wemos device outside metallic box of the alarm (or at least leave the door open) and see what will happen. Finally (if the above does't work) - you are using Wemos Pro (you state that) - it is possible to connect external antenna to amplify the signals - you must resolder a resistance near SMA connector in order this to work (google that if you don't know how) - after that put the external antenna outside the metallic box. Or try to install WiFi router closer to your alarm box....

I don't have more ideas... Just try... If it still doesn't work... I don't know :(

qntris commented 4 years ago

Hi @Angel0ffDeath I am using several boards - two NodeMCUs anD one Wemos D1 Mini Pro. I am testing them while not connected to the Paradox alarm, hooked up via mini USB to the AC (stable power supply). Wifi signal is excellent, Wemos has external antenna (resoldered the resistor).

Boards are tied with reserved IPs, lease interval is 120 minutes (disconnects happen quite often so not related to this), beacon interval is 40ms (the lowest possible). Absolute M Y S T E R Y

Angel0ffDeath commented 4 years ago

@qntris - I don't know how you can test the devices without they are connected to the alarm, but ok... you know. I hope you are using 1 device at time - the connected one and the others are disconnected. According to me it is also mystery... Do you recieve actually any messages from the alarm panel until it is connected?

If you provide a trace - this will help

Also - Can you provide a picture or scheme of wiring you are using

qntris commented 4 years ago

I test their connection to the MQTT broker and if they drip it, also test the ping availability. I am testing two at a time almost, but I thin in one of the posts @maragelis mentioned it’s not a problem - they don’t have similar ID’s or so. When connected to the Paradox alarm I get messages and I can send commands as well. The wiring is exactly as per the picture that Maragelis has included to the project here in github, bit I am also testing them connected to a mini USB and into the AC adapter. It’s something else. I have 4 other ESP8266 base devices (smart sockets) and they are hooked to the main router without any disconnects (from the broker).

I am not exactly sure how to do tracing - besides setting 1 and precompiling what else I need to do? Where do I look for the actual traces?

Angel0ffDeath commented 4 years ago

@qntris - You don't need to recompile - just post "trace=1" in "in topic" (probably paradoxdCTL/in if you didn't change it) then connect your laptop via USB cable to the micro USB port of Wemos/NodeMCU. Now you can open PuTTY sesion (serial) and record the log (trace). Mean while you can issue some commands. If you have some home automation system this is already integrated - you can send the log file also

qntris commented 4 years ago

@Angel0ffDeath , I have posted "trace=1" to the "in" topic and the Wemos' light flashes after sending it, so the connection is good. However hooking up to the serial port in Putty does not show anything - the connection to the serial port is established, but no traces are displayed on the terminal screen. I am using Home Assistant for home automation, can you give me some more hints for the traces?

maragelis commented 4 years ago

Some Chinese wemos modules have been known to have problems with the program. I have some suggestions. 1.) keep it on 2.4.1 core. 2.) use a good quality module. 3.) Only enable the features you really need. 4.) use the master branch of the code

To use trace you have to use swap serial=1 and connect the panel to D13 D15, or use a serial ftdi module connected to D8 D7 when Using hardware serial.

Know bug: there seems to be a problem with heavy systems (more then 16 zones) with high zone traffic. I believe the wemos can’t keep up with the rate of messages. Disable hassio/HomeKit/send all events flags. And check mqtt for steady flow of messages.

qntris commented 4 years ago

Hi @maragelis, it’s not a faulty module or busy system. Everything’s fine when I keep pinging the device from the AP in the other room or if I connect it to the AP in the other room. It’s something with the router settings but I have checked e v e r y option - no reason for that behavior. Hoped that tracing will help. As I don’t have FTDI adapter - is there any other way - via miniusb cable or commands OTA?

EDIT: I’ve tried with several boards, use the master branch and tried with the libraries from the readme. Don’t have anything else turned on besides hassio (per default),

Angel0ffDeath commented 4 years ago

This is trace function from the code: void trc(String msg){ if (TRACE) { Serial1.println(msg); // sendMQTT(root_topicStatus,msg); } } uncomment sendMQTT... and comment Serial1.print

Then you should receive trace messages in status topic. And of course you should first publish TRACE=1 or compile with that option on

qntris commented 4 years ago

Hi @Angel0ffDeath , I made the change that you have proposed, adding "true" (without quotes) as a third argument as it wouldn't compile otherwise. Below is the output in the "status" topic (changed the in topic to tracein to not interfere with my main board that is connected to the Paradox). I am testing this one without it being connected to the alarm module, just want to make sure that the connection to the MQTT broker is stable.

Message 50 received on paradoxdCTL/status at 1:08 PM: try again in 5 seconds QoS: 0 - Retain: false

Message 49 received on paradoxdCTL/status at 1:08 PM:
0
QoS: 0 - Retain: false

Message 48 received on paradoxdCTL/status at 1:08 PM:
failed, rc=
QoS: 0 - Retain: false
Message 47 received on paradoxdCTL/status at 1:08 PM:
 try again in 5 seconds
QoS: 0 - Retain: false

Message 46 received on paradoxdCTL/status at 1:08 PM:
0
QoS: 0 - Retain: false
Message 45 received on paradoxdCTL/status at 1:08 PM:
failed, rc=
QoS: 0 - Retain: false

Message 44 received on paradoxdCTL/status at 1:08 PM:
paradoxdCTL/tracein
QoS: 0 - Retain: false

Message 43 received on paradoxdCTL/status at 1:08 PM:
subscription OK to
QoS: 0 - Retain: false

Message 42 received on paradoxdCTL/status at 1:08 PM:
{
    "status": "Paradox connected"
}
QoS: 0 - Retain: false

Message 41 received on paradoxdCTL/status at 1:08 PM:
MQTT connected
QoS: 0 - Retain: false

Message 40 received on paradoxdCTL/status at 1:07 PM:
{
    "status": "Paradox Disconnected"
}
QoS: 0 - Retain: false
Message 39 received on paradoxdCTL/status at 1:07 PM:
failed, rc=
QoS: 0 - Retain: false

Message 38 received on paradoxdCTL/status at 1:07 PM:
 try again in 5 seconds
QoS: 0 - Retain: false
Message 37 received on paradoxdCTL/status at 1:07 PM:
0
QoS: 0 - Retain: false
Message 36 received on paradoxdCTL/status at 1:07 PM:
failed, rc=
QoS: 0 - Retain: false
Message 35 received on paradoxdCTL/status at 1:07 PM:
paradoxdCTL/tracein
QoS: 0 - Retain: false
Message 34 received on paradoxdCTL/status at 1:07 PM:
subscription OK to
QoS: 0 - Retain: false
Message 33 received on paradoxdCTL/status at 1:07 PM:
{
    "status": "Paradox connected"
}
QoS: 0 - Retain: false
Message 32 received on paradoxdCTL/status at 1:07 PM:
MQTT connected
QoS: 0 - Retain: false
Message 31 received on paradoxdCTL/status at 1:07 PM:
{
    "status": "Paradox Disconnected"
}
QoS: 0 - Retain: false
Message 30 received on paradoxdCTL/status at 1:04 PM:
 try again in 5 seconds
QoS: 0 - Retain: false
Message 29 received on paradoxdCTL/status at 1:04 PM:
0
QoS: 0 - Retain: false
Message 28 received on paradoxdCTL/status at 1:04 PM:
failed, rc=
QoS: 0 - Retain: false
Message 27 received on paradoxdCTL/status at 1:03 PM:
 try again in 5 seconds
QoS: 0 - Retain: false
Message 26 received on paradoxdCTL/status at 1:03 PM:
0
QoS: 0 - Retain: false
Message 25 received on paradoxdCTL/status at 1:03 PM:
failed, rc=
QoS: 0 - Retain: false
Message 24 received on paradoxdCTL/status at 1:03 PM:
paradoxdCTL/tracein
QoS: 0 - Retain: false
Message 23 received on paradoxdCTL/status at 1:03 PM:
subscription OK to
QoS: 0 - Retain: false
Message 22 received on paradoxdCTL/status at 1:03 PM:
{
    "status": "Paradox connected"
}
QoS: 0 - Retain: false
Message 21 received on paradoxdCTL/status at 1:03 PM:
MQTT connected
QoS: 0 - Retain: false
Message 20 received on paradoxdCTL/status at 1:03 PM:
{
    "status": "Paradox Disconnected"
}
QoS: 0 - Retain: false
maragelis commented 4 years ago

could you try this sketch if you have a second module. https://github.com/PascalSI/ParadoxRs232toMqtt/blob/master/ParadoxAlarmSystem/ParadoxAlarmSystemOTA/ParadoxAlarmSystemOTA.ino

its a first version many years back which I am still running on my system. just give it a go.

qntris commented 4 years ago

Hi @maragelis, I will give it a try, but I have two questions:

  1. Should I use the library versions from the Readme of the current master, or others?
  2. Where should I input my mqtt-user and mqtt-password - I have created a separate user in HomeAssistant that I use in all my mqtt clients for authentication.
maragelis commented 4 years ago

add them in the sketch ino file. just sniff mqtt messages.

Angel0ffDeath commented 4 years ago

@qntris - According to what I see from the trace, I think your Wemos D1 Mini Pro is going to sleep mode to save energy. Probably on your board this is the default mode. In order to prevent this you should switch to non-sleep mode I don't have Wemos Pro, but probably here @maragelis can help Sleep mode turns off all communication - including wifi

You can check this https://www.mischianti.org/2019/11/21/wemos-d1-mini-esp8266-the-three-type-of-sleep-mode-to-manage-energy-savings-part-4/

And try to disable autosleep mode and see what will happen

qntris commented 4 years ago

add them in the sketch ino file. just sniff mqtt messages.

Hi @maragelis, I managed to compile and upload it, but the wifi network that I usually connect to in order to enter my wifi credentials is not showing up.

@Angel0ffDeath I also think it is something with the sleep mode and the beacon interval, that is probably lower on the access point (no option to see it in the config page) versus the router (40ms is the lowest there). @maragelis is there some way we can try to add no sleep functionality so that I can test it that way?

Angel0ffDeath commented 4 years ago

@qntris - In setup function just add: wifi_set_sleep_type(NONE_SLEEP_T);

This will disable sleep mode. It should be added after setup_wifi(); You can also add it at the end of setup_wifi() function.

To use the sketch @maragelis asks you: if (!wifiManager.autoConnect("PARADOXController_AP", "")) {

The network is PARADOXController_AP

Angel0ffDeath commented 4 years ago

@qntris Any success??? I'm interested...

qntris commented 4 years ago

When I unplug my TP-Link RE305 range extender it is stable (using the latest master). If I have the RE305 plugged and constantly pinging the Wemos it is also stable. Why this happens, to this day, I have not found out. All other ESP8266 devices at home work well regardless of the setup.

Angel0ffDeath commented 4 years ago

@qntris Well then it seems the problem is between Wemos D1 mini Pro which you are using for the alarm and range extender. What type are the others ESP8266? Try to use the same type... I am not familiar with the extender you are using. Try to remove everything which extender could change or overwrite during establishing of the connection, i.e. give permanent IP address by reserving with MAC address (if not done), check again lease times..... I think you have some access to the extender via lan cable to set some options - check extender options and configuration as well - not only the router. I don't know what else... If without extender it works good then as I said the problem is in extender or communication between both devices.

marianomd commented 4 years ago

I am also with MG5050 and no problems here.

I read that Paradox serial port is TTL 5v. Shouldn't a ttl level shifter be needed?

Mine is a MG5000.

Angel0ffDeath commented 4 years ago

@marianomd Correct it is 5V TTL, however Wemos D1 Mini tolerates 5V on input so you don't need TTL logic level shifter unless you are using some other board which is not 5V tolerable

marianomd commented 4 years ago

Great, thanks.