Add SM100 EMS-Plus device

[S-Przybylski]

Is your feature request related to a problem? Please describe. I want to investigate the EMS and EMS-Plus messages for my solar module SM100 device in future.

Describe the solution you'd like Can you please update the device table for my SM100? ... 0x30 -> 0x09, type 0x02 telegram: 30 09 02 00 A3 15 04 (CRC=25), #data=3 <--- Version(0x02) received Unrecognized device found. TypeID 0x02, ProductID 163, Version 21.04

Additional context I assume that the implementation of SM10 is not true generic one. My current log shows messages of a SM10, but in opposite direction the device is unknown... (i do not have a SM10 connected): (00:25:56.914) 0x09 -> SM10, type 0x96 telegram: 09 B0 96 00 07 (CRC=00), #data=1 (00:25:56.967) 0x30 -> 0x09, type 0x96 telegram: 30 09 96 00 FF 18 1E 0A 02 50 28 (CRC=B5), #data=7 (00:25:56.979) 0x09 -> SM10, type 0x00 telegram: 09 B0 FF 00 FF 00 01 (CRC=0C), #data=3

The autodetect shows the following: Started scan on EMS bus for known devices Boiler found. Model Buderus GB172/Nefit Trendline (TypeID:0x08 ProductID:123 Version:04.09) Device found. Model BC25 Base Controller with TypeID 0x09, ProductID 125, Version 01.06 Unrecognized device found. TypeID 0x02, ProductID 163, Version 21.04

[DarthMob]

Hi @S-Przybylski I have a SM50 (Buderus Link) - maybe the Message header is different? However during the logfile capute i received updated solar values in mqtt. The information must be there?

[proddy]

@DarthMob it's probably an issue in how the JSON is rendered. Could you add a line in ems-esp.cpp to print out the real value before it's sent via MQTT. So somewhere around line #765 after the if (EMS_Other.SM) {

add myDebug("!! value = %d", EMS_Other.SMEnergyTotal);

then upload, wait a while, do a publish and see what the value is. Hopefully its not negative!

[DarthMob]

@proddy thanks for the info. I tried doing that with version 1.8 (had to update PIO to v4) but now i have a new issue: -> as soon as i exit serial mode and ems-esp tries to read the EMS bus i get frequent reboots of the system!

Setup with bbqkees board is exactly the same! I tried everything (erasing flash, compiling without the debug line, fresh 1.8 download...)

as you can see here from the PING there is no stable connection when reading from EMS. After "set serial on" everything is fine (done in the middle of the picture)

version 1.7 worked fine (except the issue with "-"kWh).. any ideas?

[proddy]

@DarthMob that's really strange indeed and can't think what is causing this. If you can't get to the console then you won't be able to see the stack dump to debug the cause or even use system to see whether its an ESP reset or a wifi disruption. Does it happen immediately after restarting? Or during the telnet session? Also I assume this is Wemos D1 mini with 4MB flash?

Some things you could try

1. Using platform = ${common.arduino_core_2_4_2} in the ini file and rebuilding
2. Grab 1.7.0 which you know works (bottom left of PlatformIO window click and select the 1.7.0 tag and rebuild after updating the libraries

[susisstrolch]

I have to admit that I have the same problem. Max. uptime was 1500s - mostly I only see about 30s before reboot. Still investigating...

Von meinem iPad gesendet

Am 16.06.2019 um 14:35 schrieb Proddy notifications@github.com:

@DarthMob that's really strange indeed and can't think what is causing this. If you can't get to the console then you won't be able to see the stack dump to debug the cause or even use system to see whether its an ESP reset or a wifi disruption. Does it happen immediately after restarting? Or during the telnet session? Also I assume this is Wemos D1 mini with 4MB flash?

Some things you could try

Using platform = ${common.arduino_core_2_4_2} in the ini file and rebuilding Grab 1.7.0 which you know works (bottom left of PlatformIO window click and select the 1.7.0 tag and rebuild after updating the libraries — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[proddy]

that's not good. There are some reports of people with WiFi issues since the 2.5.0 core, maybe its related? https://github.com/esp8266/Arduino/issues/5908 https://github.com/esp8266/Arduino/issues/2795

[susisstrolch]

Here‘s an excerpt from my mqtt server. ˋˋˋ setstate MQTT2_ems_esp uptime: 63s setstate MQTT2_ems_esp 2019-06-16 16:21:18 ServiceCode 0H setstate MQTT2_ems_esp 2019-06-16 16:21:18 ServiceCodeNumber 0 setstate MQTT2_ems_esp 2019-06-16 16:21:18 boilTemp 20.4 setstate MQTT2_ems_esp 2019-06-16 16:21:18 burnGas off setstate MQTT2_ems_esp 2019-06-16 16:21:18 burnStarts 138298 setstate MQTT2_ems_esp 2019-06-16 16:21:18 burnWorkMin 520179 setstate MQTT2_ems_esp 2019-06-16 16:21:18 curBurnPow 0 setstate MQTT2_ems_esp 2019-06-16 16:21:18 curFlowTemp 20.4 setstate MQTT2_ems_esp 2019-06-16 16:21:18 fanWork off setstate MQTT2_ems_esp 2019-06-16 16:21:18 heatPmp off setstate MQTT2_ems_esp 2019-06-16 16:21:18 heatWorkMin 518615 setstate MQTT2_ems_esp 2019-06-16 16:21:18 heating_active 0 setstate MQTT2_ems_esp 2019-06-16 16:21:18 ignWork off setstate MQTT2_ems_esp 2019-06-16 16:21:18 outdoorTemp 23.9 setstate MQTT2_ems_esp 2019-06-16 16:21:18 pumpMod 0 setstate MQTT2_ems_esp 2019-06-16 16:21:18 selBurnPow 100 setstate MQTT2_ems_esp 2019-06-16 16:21:18 selFlowTemp 11 setstate MQTT2_ems_esp 2019-06-16 18:28:52 shower_alert 0 setstate MQTT2_ems_esp 2019-06-16 18:28:52 shower_timer 0 setstate MQTT2_ems_esp 2019-06-16 18:28:52 start start setstate MQTT2_ems_esp 2019-06-16 18:28:52 status online setstate MQTT2_ems_esp 2019-06-15 13:22:27 tapwater_active 0 setstate MQTT2_ems_esp 2019-06-16 16:21:18 uptime 63 setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWActivated off setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWCirc off setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWComfort Hot setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWHeat off setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWSelTemp 60

ˋˋˋ As you cam See it mostly stalls after sending the shower info... Maybe connection problems with the mqtt Server?

[DarthMob]

i will try, however i have absolutely no wifi issue while serial console is "on" (i'm connected trough wifi+telnet !!). only after ems-esp goes into reading the EMS bus (serial mode off) the connection loss appears

PS: i have a nodemcuv2 board

[proddy]

@DarthMob ok, one other thing to try is set listen_mode on which will disable all Tx. You should still see data coming in (make sure serial is off). If it works then it means the new Tx code is causing the crash. I'll work on extra debugging tonight

[proddy]

@susisstrolch could you also try with set listen_mode on to see if your device stays on longer than 1500s. This disables the Tx logic. Also if you do a set publish_time 0 this will disable the MQTT posts. I'm really trying to isolate what is causing the crashes!

[d-a-v]

You can enable software serial to enable logs on some other pins while hardware serial is in use. You can also move hardware serial to other pins and still use regular pins with software serial. You won't get core logs with software serial, but you can still enable yours. Check this Don't use core 2.5.0, try 2.5.2 instead. (Sorry for intrusion)

[DarthMob]

@proddy it works! with 1.8 and listen_mode "on" everything is working - so Tx seems to be the problem?

i also used the "myDebug("!! value = %d", EMS_Other.SMEnergyTotal);" as you requested and after publish i get the following value

publish
!! value = -32768

:-(

[proddy]

@DarthMob ok so there is an endless loop in the new Tx code that causes the WDT resets. I'll have to debug this with @susisstrolch. Strange this I don't experience this on my setup.

The SMEnergyTotal is captured via sending a request, so since Tx is disabled (with the listen_mode) it'll always be a negative value. I'll make a quick modification to suppress it.

[susisstrolch]

I see something completely different - no endless loop, but

[CRASH] Last crash was 0 days 0 hours 0 minutes 8 seconds since boot time
[CRASH] Reason of restart: 2 - Fatal exception
[CRASH] Exception cause: 9 - LoadStoreAlignmentCause
[CRASH] epc1=0x40223e72 epc2=0x00000000 epc3=0x00000000
[CRASH] excvaddr=0x7e032096 depc=0x00000000
[CRASH] sp=0x3ffffe90 end=0x3fffffc0
>>>stack>>>

clean build from current master, no personal modifications, running on ESP8266-12F

[susisstrolch]

cont... which translates to

stack:
0x40224020: sflags(char const*, fs::OpenMode&, fs::AccessMode&) at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/FS.cpp:390
0x3fff4a5c: ?? at /home/gauchard/dev/esp8266/esp8266/tools/sdk/lwip2/builder/glue-lwip/esp-dhcpserver.c:31
0x402113c8: uart_set_debug at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/uart.cpp:900
0x402117f0: malloc at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/umm_malloc/umm_malloc.cpp:1685
0x4022d20a: strstr at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/string/strstr.c:80
0x40209737: MyESP::myDebug_P(char const*, ...) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:134
0x4022d1f8: strstr at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/string/strstr.c:69
0x40246fb8: umm_info at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/umm_malloc/umm_malloc.cpp:1088
0x4020fb00: pinMode at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_wiring_digital.cpp:60 (discriminator 4)
0x4024f9d5: system_station_got_ip_set at ??:?
0x4020d0d0: MyESP::begin(char const*, char const*, char const*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:1803
0x4020aa5c: MyESP::showSystemStats() at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:1154 (discriminator 9)
0x3ffe8304: ?? at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/reent/impure.c:23
0x40227977: strcasecmp at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/string/strcasecmp.c:54
0x4020c9e1: MyESP::_changeSetting(unsigned char, char const*, char const*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:721
0x4021813e: ClientContext::_write_from_source(DataSource*) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266WiFi/src/include/ClientContext.h:445
0x4020cfbd: MyESP::_wifiCallback(justwifi_messages_t, char*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:210
0x3fff4a1c: ?? at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/StackThunk.cpp:34
0x4020d090: MyESP::begin(char const*, char const*, char const*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:1790
0x40205cbc: _GLOBAL__sub_I_ds18 at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:19
  \-> inlined by: _GLOBAL__sub_I_ds18 at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:1745
0x4020f088: crc_update$constprop$0 at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_eboot_command.cpp:28
0x401006f1: cont_wrapper at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/cont.S:82

Looks like arduino library... (btw, I'm using your latest platformio.ini-example)...

[proddy]

@susisstrolch It's hard to see from the stack what caused what - I've always found it slightly misleading. Did this crash occur after a telnet command (like a setting change or a 'system')? Could it be the Telnet client like last time? Can you reproduce it also via the Serial? Running in a serial monitor will also dump the stack. You could also try 'crash test 1' which causes a crash and then compare whether the stack is pointing to the right code. For some reason I just don't trust it 100%. Anyways. need something reproducible so I can home in on the root cause. I've been running 1.8 for 4 days no with no outages.

[susisstrolch]

I've connected the Tx/Rx (GPIO1, GPIO3) to an ESPlink adapter for monitoring purposes. But I don't see the crash dump. Also, master doesn't have the 'crash test 1' command...

[susisstrolch]

argh... the stackdump file is hardcoded in the script?

[proddy]

crash test was added back to to the dev. How I use the stackdump is by copying the stack <<</>>> section and pasting into the file stackdmp.txt. Then from my PlatformIO right-click on the analyze_stackdmp.py file and hit run.

[susisstrolch]

Jep... but that won't work on linux because of two flaws a) there's nothing like .exe b) scripts/analyze_stackdmp.py has a trailing space in calling decode.py

So, here's the latest coredump. For me it seems to be an issue with the runtime (especially mDNS):

stack:
0x40223128: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_A(IPAddress, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1387
0x40208de4: OneWire::write(unsigned char, unsigned char) at ~/Development/esp8266/EMS-ESP-Boiler/.pio/libdeps/debug/OneWire_ID1/OneWire.cpp:259
0x40223180: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_PTR_IP4(IPAddress, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1408
0x40220000: esp8266::MDNSImplementation::MDNSResponder::_updateProbeStatus() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/PolledTimeout.h:185
  \-> inlined by: esp8266::MDNSImplementation::MDNSResponder::_updateProbeStatus() at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Control.cpp:1117
0x40223e6a: ArduinoJson6110_00000::JsonDeserializer<ArduinoJson6110_00000::ArduinoStreamReader, ArduinoJson6110_00000::StringCopier>::current() at ~/Development/esp8266/EMS-ESP-Boiler/.pio/libdeps/debug/ArduinoJson_ID64/src/ArduinoJson/Json/JsonDeserializer.hpp:51
0x402232c5: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_PTR_TYPE(esp8266::MDNSImplementation::MDNSResponder::stcMDNSService&, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1451
0x40223355: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_TXT(esp8266::MDNSImplementation::MDNSResponder::stcMDNSService&, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1490
0x4024760c: ppPeocessRxPktHdr at ??:?
0x40210250: SPIFFSFileImpl::size() const at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/spiffs_api.h:408
0x40250029: wifi_set_broadcast_if at ??:?
0x4020d794: fs::File::peek() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/FS.cpp:76
0x4020ad05: MyESP::_consoleShowHelp() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/WString.h:272
  \-> inlined by: ?? at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/WString.h:211
  \-> inlined by: MyESP::_consoleShowHelp() at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:530
0x40223e93: ArduinoJson6110_00000::JsonDeserializer<ArduinoJson6110_00000::ArduinoStreamReader, ArduinoJson6110_00000::StringCopier>::canBeInNonQuotedString(char) at ~/Development/esp8266/EMS-ESP-Boiler/.pio/libdeps/debug/ArduinoJson_ID64/src/ArduinoJson/Json/JsonDeserializer.hpp:313 (discriminator 3)
0x40225578: esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter::findCachedDomainOffset(void const*, bool) const at ??:?
0x40225578: esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter::findCachedDomainOffset(void const*, bool) const at ??:?
0x40225578: esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter::findCachedDomainOffset(void const*, bool) const at ??:?
0x401006cd: cont_continue at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/cont.S:51
0x40223f0f: EspClass::magicFlashChipMode(unsigned char) at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/Esp.cpp:359
0x402228b0: esp8266::MDNSImplementation::MDNSResponder::_readRRDomain_Loop(esp8266::MDNSImplementation::MDNSResponder::stcMDNS_RRDomain&, unsigned char) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:809
0x4021fc9c: esp8266::MDNSImplementation::MDNSResponder::_parseQuery(esp8266::MDNSImplementation::MDNSResponder::stcMDNS_MsgHeader const&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Control.cpp:376 (discriminator 1)
0x402205e2: esp8266::MDNSImplementation::MDNSResponder::_processAnswers(esp8266::MDNSImplementation::MDNSResponder::stcMDNS_RRAnswer const*) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Control.cpp:715
0x40100326: millis at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_wiring.cpp:186
0x4020ae6c: MyESP::_consoleShowHelp() at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:566
0x3fff2a7c: ?? at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:63
0x4020d675: EspClass::getFreeSketchSpace() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/Esp.cpp:535
0x3fff2a7c: ?? at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:63
0x40205d90: _process_UBAParametersMessage(_EMS_RxTelegram*) at ~/Development/esp8266/EMS-ESP-Boiler/src/ems.cpp:1104
0x4020f7d8: init at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_wiring.cpp:214
0x3ffe9720: ?? at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_main.cpp:55
0x401006e9: cont_wrapper at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/cont.S:81

[proddy]

it was a simple fix to get it working with linux python scripts/decoder_linux.py -s -e .pio/build/debug/firmware_d1_mini.elf scripts/stackdmp.txt

[susisstrolch]

@proddy I've pushed some changes to https://github.com/susisstrolch/EMS-ESP/tree/dev, trying to fix reboot problem. Major change:

• emsuart.cpp check for EMSbus break during send add a loopcount to avoid endless wait if send char doesn't get echoed emsuart_tx_buffer returns a 32bit result, holding Tx result, BRK and watchdog

• debug output for ems.cpp show 'poll us' enable the 'answer poll request'

When sending to EMSbus I see a EMSbus BRK approx. 10 bittimes after sending the 3rd byte to the EMSbus:

ems_parseTelegram: poll us
ems_parseTelegram: emsuart_tx_poll()
ems_parseTelegram: poll us
ems_parseTelegram: emsuart_tx_poll()
(02:27:19.731) 08 00 18 00 0B 00 D3 64 00 00 00 00 00 83 00 7D 00 80 00 00 00 FF 30 48 00 00 FF 00 00 83 00 54
(02:27:20.012) 08 00 18 1B 00 00 00 00 00 00 00 0B 00 00 FF
ems_parseTelegram: poll us
(02:27:20.944) Sending read of type 0x16 to 0x08: telegram: 0B 88 16 00 20 (CRC=EC) #data=224
_ems_sendTelegram: BRK from EMS Busmaster [3,10]!
ems_parseTelegram: poll us

Because I can't connect the logic analyzer it's hard for me to see what really happens. Maybe my hardware is faulty... So, can you try my dev build to see if the same happens on your hardware?

[proddy]

nice, I'll check it out tonight and hook up the logic analyzer. Noticed the #data=224. The buffer is only 32 bytes so this may be causing the pointer/memory exceptions. Perhaps but a test in emsuart to only add to the buffer if the length <= 32.

[susisstrolch]

Hmm, not sure where it gets the #data=224 from.. The 'BRK from EMS Busmaster [3,10]' means break while sending 3rd byte, 10 bit-times after feeding the fifo. At this point we only have send src, dst and type.

[proddy]

@susisstrolch testing your dev build and I'm not seen any errors like you get. Seems to work fine, but then again version 1.7 and 1.8 are also stable with me. Maybe it's the wifi speed? Does changing EMSESP_DELAY in ems-esp.cpp make a difference perhaps?

[susisstrolch]

So I‘ll Check my hardware...

Sent by mobile device

Am 21.06.2019 um 18:48 schrieb Proddy notifications@github.com:

@susisstrolch testing your dev build and I'm not seen any errors like you get. Seems to work fine, but then again version 1.7 and 1.8 are also stable with me. Maybe it's the wifi speed? Does changing EMSESP_DELAY in ems-esp.cpp make a difference perhaps?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[S-Przybylski]

dear all i want to close the issue. Please set up a new dedicated issue for existing problems instead using this issue (like i have done by now). Thanks