1technophile / OpenMQTTGateway

MQTT gateway for ESP8266 or ESP32 with bidirectional 433mhz/315mhz/868mhz, Infrared communications, BLE, Bluetooth, beacons detection, mi flora, mi jia, LYWSD02, LYWSD03MMC, Mi Scale, TPMS, BBQ thermometer compatibility & LoRa.
https://docs.openmqttgateway.com
GNU General Public License v3.0
3.62k stars 796 forks source link

Exception when trying to enable external decoder #1896

Closed amidarius closed 5 months ago

amidarius commented 9 months ago

Describe the bug Executing mosquitto_pub -t home/OpenMQTTGateway02/commands/MQTTtoBT/config -m '{"extDecoderEnable":true}' gateway crashes.

To Reproduce Steps to reproduce the behavior:

  1. Publish to home/OpenMQTTGateway02/commands/MQTTtoBT/config this payload {"extDecoderEnable":true}
  2. Gateway crashes. This is the log :
...
N: Connected to broker
N: Send on /SYStoMQTT msg {"uptime":857,"version":"7a3fae","disc":true,"ohdisc":false,"env":"esp32-m5stick-c-ble","freemem":81136,"mqttp":"1883","mqtts":false,"msgprc":95,"msgblck":0,"maxq":4,"minmem":22860,"tempc":55,"freestck":2096,"eth":false,"rssi":-43,"SSID":"amidar2","BSSID":"64:13:6C:1B:16:A0","ip":"192.168.44.67","mac":"D8:A0:1D:50:F4:EC","lowpowermode":0,"m5batvoltage":4.1426,"m5batcurrent":0,"m5vinvoltage":0.0034,"m5vincurrent":0,"m5vbusvoltage":4.879,"m5vbuscurrent":120,"m5tempaxp":45.7,"m5batpower":0,"m5batchargecurrent":0,"m5apsvoltage":4.7922,"modules":["WebUI","IR","BT","GPIOInput"]}
N: Send on /BTtoMQTT msg {"bleconnect":true,"interval":55555,"adaptivescan":true,"intervalacts":55555,"intervalcnct":3600000,"scanduration":10000,"onlysensors":false,"randommacs":false,"hasspresence":false,"prestopic":"presence/","presuseuuid":false,"minrssi":-100,"extDecoderEnable":false,"extDecoderTopic":"undecoded","filterConnectable":false,"pubadvdata":false,"pubuuid4topic":false,"ignoreWBlist":false,"presenceawaytimer":120000,"movingtimer":60000,"forcepscn":false,"tskstck":2080,"crstck":3052,"enabled":true,"scnct":13}
N: Send on /WebUItoMQTT msg {"displayMetric":true,"webUISecure":true,"displayQueue":0}
N: Scan begin
N: Device detected: 0F:10:83:71:5D:44
N: Device detected: 7B:D7:44:4D:D7:4C
N: Send on /BTtoMQTT/7BD7444DD74C msg {"id":"7B:D7:44:4D:D7:4C","rssi":-52}
N: Device detected: C4:82:E1:33:65:40
N: Send on /BTtoMQTT/C482E1336540 msg {"id":"C4:82:E1:33:65:40","name":"TY","rssi":-90}
N: Device detected: A4:C1:38:70:72:84
N: Send on /BTtoMQTT/A4C138707284 msg {"id":"A4:C1:38:70:72:84","name":"ATC_707284","rssi":-88,"brand":"Xiaomi","model":"TH Sensor","model_id":"LYWSD03MMC/MJWSD05MMC_PVVX","type":"THB","tempc":21.29,"tempf":70.322,"hum":51.76,"batt":53,"volt":2.573,"mac":"A4:C1:38:70:72:84"}
N: Device detected: 9E:7B:EF:8F:E5:15
N: Send on /BTtoMQTT/9E7BEF8FE515 msg {"id":"9E:7B:EF:8F:E5:15","rssi":-70}
N: Device detected: A4:C1:38:5D:C1:97
N: Send on /BTtoMQTT/A4C1385DC197 msg {"id":"A4:C1:38:5D:C1:97","name":"ATC_5DC197","rssi":-59,"brand":"Xiaomi","model":"TH Sensor","model_id":"LYWSD03MMC/MJWSD05MMC_PVVX","type":"THB","tempc":21.05,"tempf":69.89,"hum":52.38,"batt":96,"volt":2.872,"mac":"A4:C1:38:5D:C1:97"}
N: Found 6 devices, scan number 15 end
Guru Meditation Error: Core  1 panic'ed (LoadStoreError). Exception was unhandled.

Core  1 register dump:
PC      : 0x400d9bbf  PS      : 0x00060430  A0      : 0x800d9e0d  A1      : 0x3ffd2540
A2      : 0x40033cac  A3      : 0x3ffd2600  A4      : 0x00000000  A5      : 0x3ffd2628
A6      : 0x0000005c  A7      : 0x00000000  A8      : 0x0000000b  A9      : 0x3ffd2520
A10     : 0x0000000c  A11     : 0x0000003a  A12     : 0x00000000  A13     : 0x00000001
A14     : 0x00ff0000  A15     : 0xff000000  SAR     : 0x00000018  EXCCAUSE: 0x00000003
EXCVADDR: 0x40033cb4  LBEG    : 0x40092ab9  LEND    : 0x40092ac9  LCOUNT  : 0xfffffffe

Backtrace: 0x400d9bbc:0x3ffd2540 0x400d9e0a:0x3ffd2560 0x400d9e8f:0x3ffd2580 0x400ed599:0x3ffd2650 0x400ed6bf:0x3ffd2a20 0x401ecbdb:0x3ffd2a40 0x400f1ab5:0x3ffd2a60 0x400f053e:0x3ffd2aa0 0x4013314d:0x3ffd2ae0

ELF file SHA256: f499a23c4817b49b

Rebooting...

Expected behavior extDecoderEnable config parameter should be true after publishing apropriate payload to .../MQTTtoBT/config without Guru Meditation.

Environment :

Additional context The board is M5StickC and esp32-m5stick-c-ble firmware was uploaded.

amidarius commented 9 months ago

Found and flashed esp32dev-ble-mqtt-undecoded on another ESP32 board and i got what i wanted but, why is this separated into separate firmware and not an option inside firmware for choosen board through web install ? πŸ€”

1technophile commented 9 months ago

It should work as an option at runtime also, we will try to reproduce

amidarius commented 9 months ago

Found and flashed esp32dev-ble-mqtt-undecoded on another ESP32 board and i got what i wanted but, why is this separated into separate firmware and not an option inside firmware for choosen board through web install ? πŸ€”

Well not exactly, πŸ™„. There was undecoded topic with this firmware but without raw data ?! so i finally found the solution and it is located inside esp32dev-ble-datatest-firmware.bin firmware. Actually, this is the firmware i needed because for supported device there is decoded data and for unsupported devices there is manufacturerdata (If exist of course.) what else do you need ? 😎

image

Now when i have manufacturerdata i can decode data from still unsupported HRM. Until now and, hope to see it supported in next release of Android APP version, propagated from decoder of course. πŸ˜„

Manufacturer name: XOSS Works Model number : X2H Aliexpress link : XOSS X2 Chest Strap Heart Rate Sensor Bike Monitor Bluetooth ANT+ Wireless Health Fitness Smart Bicycle Data Tracker Manufacturer data: 04ff0161063f

So, decoded, these bytes are :

04 - length, 4 bytes of data. FF - Indicates "MANUFACTURER_SPECIFIC_DATA"

- Actual DATA -

01 - Body sensor location = chest org.bluetooth.characteristic.body_sensor_location.xml 61 - Battery level (percentage) = 97% 06 - unknown πŸ€” 3f - Heart rate = (63 bpm)

image

DigiH commented 9 months ago

Hi @amidarius

The option to show the raw advertising data along even with decoded and also with undecoded devices can also be turned on with any other binary by setting Advertisement and advanced data to true.

esp32dev-ble-datatest is basically just the esp32dev-ble firmware with this option turned on.

It would be interesting to include your XOSS X2 heart rate chest strap to Theengs Decoder though, as we have done with the Polar H10 chest strap.

Could you test and see if the name:"XOSS_X2_XXXXXX" is also being broadcast with passive scanning, or if it does require active scanning for the name to show?

DigiH commented 9 months ago

The issue with activating {"extDecoderEnable":true} remains though, but isn't really required here.

amidarius commented 9 months ago

This is the result from HumBLE explorer, just pure passive scanning, no connection inolved.

image

And yes, this is very similar device as already present H10.

amidarius commented 9 months ago

Hi @amidarius

The option to show the raw advertising data along even with decoded and also with undecoded devices can also be turned on with any other binary by setting Advertisement and advanced data to true.

Well, just tried mosquitto_pub -t home/OpenMQTTGateway02/commands/MQTTtoBT/config -m '{"pubadvdata":true}' and the result is Guru meditation again. So, basically sending anything to config is problematic ?! πŸ€” At least on esp32-m5stick-c-ble firmware.

DigiH commented 9 months ago

Thanks for the confirmation about passive scanning, but I've implemented the decoder linked to above with just the manufacturerdata length and starting octets.

All ready for merging into Theengs Decoder so that the XOSS X2 will be recognised with the next releases of OpenMQTTGateway, Theengs Gateway and the Theengs mobile App.

DigiH commented 9 months ago

Well, just tried mosquitto_pub -t home/OpenMQTTGateway02/commands/MQTTtoBT/config -m '{"pubadvdata":true}' and the result is Guru meditation again. So, basically sending anything to config is problematic ?! πŸ€” At least on esp32-m5stick-c-ble firmware.

That is really weird and needs looking into, I assume this is with the 1.7.0 release, or did you try the development binaries?

If not, could you test it with the latest development binaries at https://docs.openmqttgateway.com/dev/upload/web-install.html and let us know how commands work there with your m5stick-c.

amidarius commented 9 months ago

Yes, it was with the 1.7.0 release and i just tried with the latest development binaries. The result of mosquitto_pub -t home/OMG_ESP32_M5STICK_C_BLE_IR/commands/MQTTtoBT/config -m '{"pubadvdata":true}' is this.

Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x400d9850  PS      : 0x00060430  A0      : 0x800d9dff  A1      : 0x3ffd2370
A2      : 0x3ffd2430  A3      : 0x00000000  A4      : 0x0000000c  A5      : 0x3ffd2458
A6      : 0x0000005c  A7      : 0x00000000  A8      : 0x800d984e  A9      : 0x3ffd2350
A10     : 0x3ffd2430  A11     : 0x00000022  A12     : 0x00000000  A13     : 0x00000001
A14     : 0x00ff0000  A15     : 0xff000000  SAR     : 0x00000017  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000000  LBEG    : 0x40092ab9  LEND    : 0x40092ac9  LCOUNT  : 0xfffffffe

Backtrace: 0x400d984d:0x3ffd2370 0x400d9dfc:0x3ffd2390 0x400d9e8f:0x3ffd23b0 0x400ed599:0x3ffd2480 0x400ed6bf:0x3ffd2850 0x401ecbdb:0x3ffd2870 0x400f1ab5:0x3ffd2890 0x400f053e:0x3ffd28d0 0x4013314d:0x3ffd2910

ELF file SHA256: 52e94381987ce61b

Rebooting...

Β―\_(ツ)_/Β―

DigiH commented 9 months ago

Definitely needs looking into with the m5stick-c.

You can still install the development version again tomorrow, after the nightly build will included the development version of Theengs Decoder with the now merged XOSS X2 decoder.

Let us know if it decodes fine for you.

amidarius commented 9 months ago

You can still install the development version again tomorrow, after the nightly build will included the development version of Theengs Decoder with the now merged XOSS X2 decoder.

Let us know if it decodes fine for you.

It seems like it is correctly decoded. πŸ‘Œ

image

DigiH commented 9 months ago

Thanks for the confirmation about the decoder @amidarius

Keeping this issue open for the m5stick-c MQTTtoBT command issues.

hallard commented 9 months ago

Hi there,

sorry to get in but I have the same issue with different target, firmware crash and fire the error on Thingpulse GW, so I tried

Definitively something introduced in decoder of V1.7

W: MQTT connection...
N: Connected to broker
N: Update check, free heap: 73436N: Update check done, free heap: 32064N: Send on /SYStoMQTT msg {"uptime":8,"version":"46160c","rgbb":255,"disc":true,"ohdisc":false,"env":"thingpulse-espgateway","freemem":75908,"mqttp":"1883","mqtts":false,"msgprc":0,"msgblck":0,"maxq":0,"minmem":69528,"tempc":58.33,"freestck":5908,"eth":false,"rssi":-58,"SSID":"CH2I-HOTSPOT","BSSID":"94:83:C4:A2:11:92","ip":"192.168.1.214","mac":"1C:9D:C2:4A:DA:08","lowpowermode":-1,"modules":["WebUI","BT"]}
N: Send on /BTtoMQTT msg {"bleconnect":true,"interval":55555,"adaptivescan":true,"intervalacts":55555,"intervalcnct":3600000,"scanduration":10000,"onlysensors":false,"randommacs":false,"hasspresence":false,"prestopic":"presence/","presuseuuid":false,"minrssi":-100,"extDecoderEnable":false,"extDecoderTopic":"undecoded","filterConnectable":false,"pubadvdata":false,"pubuuid4topic":false,"ignoreWBlist":false,"presenceawaytimer":120000,"movingtimer":60000,"forcepscn":false,"tskstck":5444,"crstck":4576,"enabled":true,"scnct":0}
N: Send on /WebUItoMQTT msg {"displayMetric":true,"webUISecure":true,"displayQueue":0}
N: Send on /RLStoMQTT msg {"latest_version":"46160c","title":"OpenMQTTGateway Nightly (TEST ONLY)","installed_version":"46160c","entity_picture":"https://github.com/1technophile/OpenMQTTGateway/raw/development/docs/img/Openmqttgateway_logo_mini_margins.png","release_summary":""}
N: Scan begin
N: Device detected: 6F:A8:32:49:ED:A1
N: Device detected: 74:38:B7:B5:22:48
N: Device detected: 50:8A:06:BC:D7:D7
N: Send on /BTtoMQTT/508A06BCD7D7 msg {"id":"50:8A:06:BC:D7:D7","name":"TY","rssi":-51}
N: Device detected: 7D:F5:6D:25:77:49
N: Device detected: F8:1B:54:2B:DC:79
N: Device detected: CD:1A:C7:01:E1:8B
N: Device detected: 0A:71:48:E5:D7:AA
N: Device detected: 5C:4C:FD:25:52:02
N: Device detected: EC:05:C4:18:45:DE
N: Device detected: 54:C9:DF:C6:88:FF
N: Device detected: 5C:E5:0C:22:98:6B
N: Device detected: 58:2D:34:12:30:BC
N: Device detected: 5E:70:2C:41:EC:5B
N: Device detected: F1:FD:88:54:BE:A6
N: Device detected: D0:71:4A:55:09:8D
N: Device detected: 84:CC:A8:7F:1B:A2
N: Device detected: C8:64:C8:B7:B3:69
N: Device detected: 5F:23:D0:DF:F9:DA
N: Send on /BTtoMQTT/EC05C41845DE msg {"id":"EC:05:C4:18:45:DE","name":"Ruuvi 45DE","rssi":-58,"brand":"Ruuvi","model":"RuuviTag","model_id":"RuuviTag_RAWv2","type":"ACEL","tempc":17.1,"tempf":62.78,"hum":50.385,"pres":985.54,"accx":1.004201,"accy":0.015691,"accz":0.078453,"volt":3.083,"tx":4,"mov":14,"seq":4150,"mac":"EC:05:C4:18:45:DE"}

Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40092113  PS      : 0x00060430  A0      : 0x800d9fc7  A1      : 0x3ffd1e10  
A2      : 0x00000000  A3      : 0x3f43efce  A4      : 0x3ffd2360  A5      : 0x3ffd2360  
A6      : 0x3ffd2360  A7      : 0x3ffebe98  A8      : 0x800d5346  A9      : 0x3ffd1de0  
A10     : 0x00000000  A11     : 0x3f40218d  A12     : 0x00000010  A13     : 0x00000001  
A14     : 0x00000021  A15     : 0x3fff2938  SAR     : 0x00000011  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000000  LBEG    : 0x4009182c  LEND    : 0x40091837  LCOUNT  : 0x00000000  

Backtrace: 0x40092110:0x3ffd1e10 0x400d9fc4:0x3ffd1e20 0x400ed7f7:0x3ffd2380 0x40122479:0x3ffd23c0
ELF file SHA256: d0cf6d08adb4311c
Rebooting...
ets Jul 29 2019 12:21:46
rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:1184
load:0x40078000,len:13192
load:0x40080400,len:3028
entry 0x400805e4

Can also happens after another sensor type (not always ruuvi)

N: Device detected: C4:45:EB:6E:0A:4A
N: Device detected: CC:E3:6D:F2:0C:74
N: Device detected: D0:71:4A:55:09:8D
N: Device detected: A8:80:55:4F:C1:FA
N: Send on /BTtoMQTT/582D341233F7 msg {"id":"58:2D:34:12:33:F7","name":"CGG_1233F7","rssi":-55,"brand":"ClearGrass/Qingping","model":"Round TH","model_id":"CGG1_ATC1441","type":"THB","tempc":17.8,"tempf":64.04,"hum":51,"batt":63,"volt":2.769,"mac":"58:2D:34:12:33:F7"}

Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40092113  PS      : 0x00060430  A0      : 0x800d9fc7  A1      : 0x3ffd1e10  
A2      : 0x00000000  A3      : 0x3f43efce  A4      : 0x3ffd2360  A5      : 0x3ffd2360  
A6      : 0x3ffd2360  A7      : 0x3fffc880  A8      : 0x800d5346  A9      : 0x3ffd1de0  
A10     : 0x00000000  A11     : 0x3f40218d  A12     : 0x00000010  A13     : 0x00000001  
A14     : 0x00000021  A15     : 0x3fffd428  SAR     : 0x00000011  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000000  LBEG    : 0x4009182c  LEND    : 0x40091837  LCOUNT  : 0x00000000  
ssalonen commented 8 months ago

@hallard , the ruuvi crash is likely due to https://github.com/1technophile/OpenMQTTGateway/issues/1905

See corresponding PR that fixes that particular crash with Ruuvi

github-actions[bot] commented 5 months ago

This issue is stale because it has been open for 90 days with no activity.

github-actions[bot] commented 5 months ago

This issue was closed because it has been inactive for 14 days since being marked as stale.