theengs / gateway

Multi platform Multi devices BLE Bluetooth to MQTT gateway leveraging Theengs Decoder
https://gateway.theengs.io
GNU General Public License v3.0
111 stars 31 forks source link

Nothing is decoded in the topic BTtoMQTT #251

Closed duylong closed 5 months ago

duylong commented 5 months ago

Hi,

I installed Theengs Gateway 1.4.0 with Docker and I have a decoding problem. Indeed, despite my efforts, no information in "servicedata" is decoded. My equipment is well taken care of.

{"name": "Flower mate", "id": "**:**:**:**:**:**", "rssi": -87, "servicedatauuid": "fe95", " servicedata": "31029800008a50628d7cc40d", "brand": "Xiaomi/VegTrug", "model": "MiFlora", "model_id": "HHCCJCY01HHCC", "type": "PLANT", "mac": "**:* *:**:**:**:**"}

Here is my console:

Creating config at /root/theengsgw.conf ...
{
     "host": "x.x.x.x",
     "pass": "",
     "user": "",
     "port": 1883,
     "publish_topic": "tgw/BTtoMQTT",
     "subscribe_topic": "tgw/BTtoMQTT/undecoded",
     "presence_topic": "tgw/presence",
     "presence": false,
     "general_presence": false,
     "publish_all": false,
     "publish_advdata": true,
     "ble_scan_time": 60,
     "ble_time_between_scans": 60,
     "tracker_timeout": 120,
     "log_level": "INFO",
     "lwt_topic": "tgw/status",
     "discovery": false,
     "hass_discovery": false,
     "discovery_topic": "homeassistant",
     "discovery_device_name": "tgw",
     "discovery_filter": "[IBEACON,GAEN,MS-CDP,APPLE_CONT,APPLE_CONTAT]",
     "scanning_mode": "active",
     "adapt": "hci1",
     "time_sync": "[]",
     "time_format": "0",
     "ble": true,
     "enable_tls": false,
     "enable_websocket": false
}
INFO:BLEGateway:Starting BLE scan
INFO:BLEGateway:Connected to MQTT broker
INFO:BLEGateway:Subscribed to tgw/BTtoMQTT/undecoded

Is there a problem? Does anyone have this issue?

DigiH commented 5 months ago

Hi @duylong

Is this the only kind of advertising data you get form your device, i.e. servicedata": "31029800008a50628d7cc40d" with the servicedata being 24 characters long?

It definitely looks like a mis-recognition as a Xiaomi/VegTrug MiFlora. What kind of device is it actually?

Possibly we need to create a new decoder for this device together with your help.

Could you also tell us if the MAC address/ID is included in the servicedata or not? It's fine to be anonymised , bnut important if it is correct or in reverse order, e.g. AABBCCDDEEFF or FFEEDDCCBBAA.

BTW, you might want to reduce your "ble_scan_time" to something between 5-10 seconds, which is more than enough to catch all available devices every 60 seconds ble_time_between_scans.

duylong commented 5 months ago

Ah I thought there was a problem in the decoding, I didn't understand what was happening.

I have several Mi Plant sensors (quite old), I don't think there are any errors on the type of devices. I anonymize by default, here is the JSON for 3 devices with their MAC address:

{"name": "Flower mate", "id": "C4:7C:8D:62:50:8A", "rssi": -78, "servicedatauuid": "fe95", "servicedata": "31029800008a50628d7cc40d" , "brand": "Xiaomi/VegTrug", "model": "MiFlora", "model_id": "HHCCJCY01HHCC", "type": "PLANT", "mac": "C4:7C:8D:62:50:8A"}
{"name": "Flower mate", "id": "C4:7C:8D:60:CA:CC", "rssi": -87, "servicedatauuid": "fe95", "servicedata": "3102980020ccca608d7cc40d" , "brand": "Xiaomi/VegTrug", "model": "MiFlora", "model_id": "HHCCJCY01HHCC", "type": "PLANT", "mac": "C4:7C:8D:60:CA:CC"}
{"name": "Flower mate", "id": "C4:7C:8D:60:F0:CE", "rssi": -92, "servicedatauuid": "fe95", "servicedata": "3102980000cef0608d7cc40d" , "brand": "Xiaomi/VegTrug", "model": "MiFlora", "model_id": "HHCCJCY01HHCC", "type": "PLANT", "mac": "C4:7C:8D:60:F0:CE"}

And yes I seem to have only 31029800008a50628d7cc40d in the servicedata.

DigiH commented 5 months ago

I have several Mi Plant sensors (quite old), I don't think there are any errors on the type of devices.

Unfortunately there are so many variants of the Mi Flora around, all with different kinds of encoding, so that yours fell into one of the other Mi Flora decoder, but not the one which actually matches your variant :( which so far, from what I'm seeing, is a HHCCJCY01 and not a HHCCJCY01HHCC.

As there also seems to be some slightly different encoding issues with the firmware, it could also well be that your quite old devices might still have an older firmware which requires a different decoding - just to make thing even more interesting ;)

Actually, the main part of the servicedata your Mi Floras are broadcasting is the actual MAC address in reverse of the device, leaving little other information to include any other data which your are interested in, i.e. "C4:7C:8D:62:50:8A" - "31029800008a50628d7cc40d" "C4:7C:8D:60:CA:CC" - "3102980020ccca608d7cc40d" "C4:7C:8D:60:F0:CE" - "3102980000cef0608d7cc40d"

so with the leading always same 31029800 this only leaves one octet in between these datasets and the octet at the end, which doesn't seem to be changing either.

Unless you might see different kind of broadcast servicedata for these devices there currently is no decodable data included.

The only way I can see these devices working in broadcasting sensible data to be decoded might be to update their firmware, but I honestly don't know if that would change anything.

Alternatively you could try some other integration, which actually connects to the devices to fetch data, but with the current BLE broadcasts, which Gateway is currently only capable of, it doesn't look possible to decode anything.

I have added a fix to the HHCCJCY01HHCC decoder so that in the future they at least won't get wrongly recognised as HHCCJCY01HHCCs.

duylong commented 5 months ago

C4:7C:8D:60:CA:CC and C4:7C:8D:60:F0:CE are outside, while C4:7C:8D:62:50:8A is inside.

And yes I added the Flower Case app because I thought I had a firmware problem and I already had the latest version. For the moment I'm not at home, I can't check with the application, but with the bluepy library, I have this:

C4:7C:8D:60:CA:CC = 8.9 °C C4:7C:8D:60:F0:CE = 9.6 °C C4:7C:8D:62:50:8A = 18.4 °C

It may be possible in a future version to query in BLE, but I understand that in my case, this is not currently possible.

DigiH commented 5 months ago

Closing this as unfortunately currently these devices are not decodable through their BLE advertising data.

duylong commented 5 months ago

A use case for example : https://github.com/jeedom/plugin-blea/blob/beta/resources/blead/devices/miflora.py