Add support for ATC's format

[eduperez]

ATC firmware can be configured to send info using "Mi-like" format, or it's own format. I think it would be great to be able to receive both.

[Ernst79]

I don't have this device myself, but a first step could be to show the output from the following command.

btmon --write hcitrace.snoop | tee hcitrace.txt

In the output, look for your sensor and copy the relevant part here, to investigate what is possible.

We will need the UUID, which is probably not 0xfe95 (= Xiaomi) and the payload.

More information on the mi-like format can be found here. https://github.com/custom-components/ble_monitor/blob/master/protocol.md

If the format is not too different, it should be possible.

[eduperez]

This is the output of btmon:

> HCI Event: LE Meta Event (0x3e) plen 29                                                        #15 [hci0] 77.644221
      LE Advertising Report (0x02)
        Num reports: 1
        Event type: Connectable undirected - ADV_IND (0x00)
        Address type: Public (0x00)
        Address: A4:C1:38:5B:0E:DF (Telink Semiconductor (Taipei) Co. Ltd.)
        Data length: 17
        Service Data (UUID 0x181a): a4c1385b0edf00e728640c3a03
        RSSI: -76 dBm (0xb4)

That should correspond to a temperature of 22.6C, an humidity of 46%, and a battery at 100%.

ATC format is quite simple and similar to Mi's (fields are in different positions and byte order, but not much else). It uses UUID 1A18, and all the info goes into the same packet. The source code from the ATC firmware where both formats are generated can be found at https://github.com/atc1441/ATC_MiThermometer/blob/master/ATC_Thermometer/ble.c

[Ernst79]

I've ordered two of these sensors, will look into it when I receive them, but this could take a while (they have to come over from china).

[rayholland]

Today i received this cheap Temp/Humidity sensor and flashed the ATC firmware. Can it be that the orginal Mi advertising format/structure broadcast message does not include battery level (because this value stays empty in Passive BLE Monitor)?

If so, then it would be great if BLE Monitor would support the ATC format.

Data format/structure should be like this:

I.e Device name: ATC_B2AA80 -> B2AA80 (last 6 digits mac address)

Service data: UUID: 0x181A Data: 0xA4C138B2AA8000FF39580BA302

A4C138B2AA80 -> HEX mac bluetooth device 00FF -> HEX temp data (int16) -> needs to be devived by /10 39 -> HEX hum. data (int8) 58 -> HEX battery percentage (int8) 0BA3 -> HEX battery voltage in mV (int16) 02 -> HEX counter (int8)

Thanks.

EDIT: I now reveice the battery value when using the 'Mi-like' format. Turned out that i just needed to wait a bit longer ;-)

[Ernst79]

Battery is reported once in an hour with stock firmware. Not sure about the custom firmware.

[rayholland]

Thanks for clarification Ernst.

[eduperez]

Can it be that the orginal Mi advertising format/structure broadcast message does not include battery level (because this value stays empty in Passive BLE Monitor)?

The ATC firmware, when configured to send "Mi-like" messages, alternates between two types of message (using the same UUID="0x95FE"): some messages have "0x0D1004" just after the MAC address, and contain the temperature and humidity, some messages have "0x0a1001" just after the MAC access, and contain the battery level (voltage is never sent).

[Ernst79]

That is the same as the original firmware, so battery should show up.

Edit, I see it already showed up for @rayholland

[rayholland]

Yeah, it makes sense they only measure the battery only once 1 hour, because i think measering the voltage is done by 'turning on/off' a voltage divider which drains the battery.

[Ernst79]

Sensors have arrived, will look into it this weekend,

[Ernst79]

@eduperez I made a branch ATC_support which adds support for sensors that broadcast in the default "custom" ATC advertising type.

So it should be able to handle both custom and Mi-Like advertisements.

Could you give it a try? Just copy __init__.py from the ATC_support branch to your custom_components/ble_monitor folder.

I will add the "voltage" measurement later.

[anilyildiz]

Hi, while you are at it, you might be interested in an improved version of the firmware initially developed by @atc1441. It can be found here. @atc1441 even recommends this version over his own for various reasons. There are more than one format for messages.

[atc1441]

+1

@pvvx version does advertise all different formarts one after each other.

[Ernst79]

I'll look into it, and will add the other format as well.

[Ernst79]

Almost there, will upload a new version tomorrow with support for the "custom" format. Also added the voltage measurement.

[Ernst79]

I've added the custom advertising format and the voltage measurement, it now supports, the ACT1441 format, custom format and Mi-Like format. For those who want to try, copy the entire ble_monitor directory from https://github.com/custom-components/ble_monitor/tree/ATC_support_v2 to you Home Assistant custom_components folder.

I have to check what happens if I put the advertisement to all, I guess it gets mixed readings from all three types of advertisements, decreasing the accuracy. So, it makes more sense if you use the custom advertisement type.

Edit. (note to myself) Using one advertisement type works, but using all as advertising type (so basically sending the advertisements in three different formats) does create an error at the moment. Error seems to be a mix-up of the battery and voltage measurement, which is in the long ATC format first voltage, than battery, while in the short format, its first battery, than voltage. Will look into that later.

[eduperez]

Just tried the v2 branch, and it works perfectly; many thanks!

[Ernst79]

Thanks, I almost solved the issue when using the all advertisements option. I hope to finish this weekend and will make a beta release.

[Ernst79]

0.9.7-beta is released with ATC advertisement support

It took a bit longer, as I tried to add the voltage sensor on the fly (when updating from mi-like to ATC formats). Unfortunately, I failed, so as a workaround, you will have to reload the integration (or restart HA) if you want to see the voltage sensor (one time only). Not a big deal, so I leave it like this. You will get a warning when a voltage sensor is recognized, after switching from Xiaomi firmware to ATC firmware (or better said from mi-like advertisements to custom ATC advertisements).

Note that using all as advertisement type in the firmware from pvvx is not the best option, as it will collect also the less accurate mi-like advertisements and combines them all, causing a decrease in accuracy. Unless you need it for something else, I suggest you select the custom advertisement type.

It works for both custom firmware from @atc1441, as well as from @pvvx

At the moment it only supports the firmware for LYWSD03MMC I notices there also is a firmware for MHO-C401. If someone want it to work with this sensor as well, please send me the advertisement format (using these instructions with report_unknow).

[clau-bucur]

I've upgraded to 0.9.7 beta, and flashed pvvx to all 3 LYWSD03MMC. Changed the advertising type to custom. None of the sensors are available, even after several restarts.

[EDIT] Switched to MI advert type and now they became available.

[Ernst79]

The mi-like advert type is the same as the default firmware, but the ATC advertisment type is "more accurate" and also gives you the voltage of the battery.

Could you try to enable report_unkown and debug logging as explained in the FAQ. It should also show unknown sensors that follow the ATC advertisement type.

[Ernst79]

I noticed that the advertisement type set to all doesn't play nice. Depending on whether the first message is mi-like or custom you won't or will get the voltage sensor. I will change the Error message to advise to use only one advertisement type and will change the level to Warning.

[Ernst79]

0.9.8 (final) has been released with support for ATC advertisement.

[eduperez]

This was working for me on the v2 branch (installed manually). However, after updating to 0.9.9 through HACS, it does not detect devices using ATC's format any longer.

Using ATC's format, this is all I see on the logs:

2021-01-31 00:43:37 DEBUG (MainThread) [custom_components.ble_monitor] Initializing BLE Monitor integration (YAML): {'rounding': True, 'decimals': 2, 'period': 60, 'log_spikes': False, 'use_median': False, 'active_scan': False, 'hci_interface': [0], 'batt_entities': True, 'discovery': False, 'restore_state': False, 'report_unknown': False, 'devices': [{'mac': 'A4:C1:38:5B:0E:DF', 'name': 'mijia_1', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:02:7E:11', 'name': 'mijia_2', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:9C:A9:57', 'name': 'mijia_3', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:D4:72:EA', 'name': 'mijia_4', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:8E:B1:28', 'name': 'mijia_5', 'temperature_unit': '°C'}], 'is_flow': False, 'ids_from_name': True}
2021-01-31 00:43:37 DEBUG (MainThread) [custom_components.ble_monitor.config_flow] async_step_import: {'rounding': True, 'decimals': 2, 'period': 60, 'log_spikes': False, 'use_median': False, 'active_scan': False, 'hci_interface': [0], 'batt_entities': True, 'discovery': False, 'restore_state': False, 'report_unknown': False, 'devices': '--Devices--', 'is_flow': False, 'ids_from_name': True}
2021-01-31 00:43:37 DEBUG (MainThread) [custom_components.ble_monitor.config_flow] async_step_user: {'rounding': True, 'decimals': 2, 'period': 60, 'log_spikes': False, 'use_median': False, 'active_scan': False, 'hci_interface': [0], 'batt_entities': True, 'discovery': False, 'restore_state': False, 'report_unknown': False, 'devices': '--Devices--', 'is_flow': False, 'ids_from_name': True}
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] Initializing BLE Monitor entry (config entry): <homeassistant.config_entries.ConfigEntry object at 0x7f9ea26220>
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: domain {'rounding': True, 'decimals': 2, 'period': 60, 'log_spikes': False, 'use_median': False, 'active_scan': False, 'hci_interface': [0], 'batt_entities': True, 'discovery': False, 'restore_state': False, 'report_unknown': False, 'devices': [{'mac': 'A4:C1:38:5B:0E:DF', 'name': 'mijia_1', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:02:7E:11', 'name': 'mijia_2', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:9C:A9:57', 'name': 'mijia_3', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:D4:72:EA', 'name': 'mijia_4', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:8E:B1:28', 'name': 'mijia_5', 'temperature_unit': '°C'}], 'is_flow': False, 'ids_from_name': True}
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: {'rounding': True, 'decimals': 2, 'period': 60, 'log_spikes': False, 'use_median': False, 'active_scan': False, 'hci_interface': [0], 'batt_entities': True, 'discovery': False, 'restore_state': False, 'report_unknown': False, 'devices': [{'mac': 'A4:C1:38:5B:0E:DF', 'name': 'mijia_1', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:02:7E:11', 'name': 'mijia_2', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:9C:A9:57', 'name': 'mijia_3', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:D4:72:EA', 'name': 'mijia_4', 'temperature_unit': '°C'}, {'mac': 'A4:C1:38:8E:B1:28', 'name': 'mijia_5', 'temperature_unit': '°C'}], 'is_flow': False, 'ids_from_name': True}
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] HCI interface is [0]
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] Spawning HCIdump thread
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] HCIdump thread: Init
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] 0 encryptors mac:key pairs loaded.
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] whitelist: [A4:C1:38:5B:0E:DF, A4:C1:38:02:7E:11, A4:C1:38:9C:A9:57, A4:C1:38:D4:72:EA, A4:C1:38:8E:B1:28]
2021-01-31 00:43:38 DEBUG (MainThread) [custom_components.ble_monitor] 5 whitelist item(s) loaded.
2021-01-31 00:43:38 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Run
2021-01-31 00:43:38 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-01-31 00:43:38 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2021-01-31 00:43:40 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Starting binary sensor entry startup
2021-01-31 00:43:40 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialization
2021-01-31 00:43:40 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialized
2021-01-31 00:43:40 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary sensor entry setup finished
2021-01-31 00:43:40 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary entities updater loop started!
2021-01-31 00:43:43 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Starting measuring sensor entry startup
2021-01-31 00:43:43 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialization
2021-01-31 00:43:43 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialized
2021-01-31 00:43:43 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Measuring sensor entry setup finished
2021-01-31 00:43:43 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Entities updater loop started!
2021-01-31 00:44:41 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 0 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2021-01-31 00:44:44 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 0 MiBeacon BLE ADV messages processed for 0 measuring device(s).
2021-01-31 00:44:44 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2021-01-31 00:44:44 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2021-01-31 00:44:44 DEBUG (Thread-3) [custom_components.ble_monitor] 775 HCI events processed for previous period.
2021-01-31 00:44:44 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Run
2021-01-31 00:44:44 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-01-31 00:44:44 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: start main event_loop

When I change back to Mi-like format, these lines apear, and it starts working again:

2021-01-31 00:49:17 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Sensor device with mac address A4:C1:38:5B:0E:DF has the following settings. Name: mijia_1.
2021-01-31 00:49:17 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Temperature sensor with mac address A4:C1:38:5B:0E:DF is set to receive data in °C
2021-01-31 00:49:17 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Sensor device with mac address A4:C1:38:5B:0E:DF has the following settings. Name: mijia_1.
2021-01-31 00:49:17 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Sensor device with mac address A4:C1:38:5B:0E:DF has the following settings. Name: mijia_1.
2021-01-31 00:49:17 DEBUG (MainThread) [custom_components.ble_monitor.sensor] async_added_to_hass called for ble temperature mijia_1
2021-01-31 00:49:17 DEBUG (MainThread) [custom_components.ble_monitor.sensor] async_added_to_hass called for ble humidity mijia_1
2021-01-31 00:49:17 DEBUG (MainThread) [custom_components.ble_monitor.sensor] async_added_to_hass called for ble battery mijia_1
2021-01-31 00:49:42 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 0 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2021-01-31 00:49:45 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 1 MiBeacon BLE ADV messages processed for 1 measuring device(s).
2021-01-31 00:49:45 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2021-01-31 00:49:45 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2021-01-31 00:49:45 DEBUG (Thread-3) [custom_components.ble_monitor] 856 HCI events processed for previous period.
2021-01-31 00:49:45 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Run
2021-01-31 00:49:45 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-01-31 00:49:45 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: start main event_loop

[Ernst79]

There will be an update this afternoon, where we add support for Bluetooth 5 dongles that were not able to read ATC sensors. I propose you try this update first. It will be released later today. If the issue still remains, let me know (preferably in a new issue)

[Ernst79]

@eduperez 0.9.11-beta is released yesterday with some improvements for ATC firmware. Could you try it with that version.

If not, please create a new issue and we will look into it together.

[clau-bucur]

Just a side note, I installed 0.9.11-beta and it reads all my ATC sensors with the Bluetooth 5 dongle. Thank you.