Device S10 Status is not send via MQTT when received via webhook

sVnsation commented 2 weeks ago

Hello, when status data like workingStatus, taskType, battery, etc is received via webhook it is not send to MQTT.

See example: Data received 09:24 am via webhook like "taskType":"cleanAll" or "workingStatus":"Clearing" is missing in the MQTT-Payload.

MQTT Messages:

`Received 08:31:29 => POLLING

QoS: 0
Payload:

deviceId: XXXXXXX
deviceType: WoSweeperOrigin
workingStatus: ChargeDone
onlineStatus: online
battery: 100
waterBaseBattery: 100
taskType: standBy

Received 09:24:25 => WEBHOOK

QoS: 0
Payload:

deviceType: Robot Vacuum Cleaner S10
deviceId: XXXXXXX

Received 09:24:33 => WEBHOOK

QoS: 0
Payload:

deviceType: Robot Vacuum Cleaner S10
deviceId: XXXXXXX

Received 09:24:35 => WEBHOOK

QoS: 0
Payload:

deviceType: Robot Vacuum Cleaner S10
deviceId: XXXXXXX

Received 09:24:45 => WEBHOOK

QoS: 0
Payload:

deviceType: Robot Vacuum Cleaner S10
deviceId: XXXXXXX

Received 09:25:35 => WEBHOOK

QoS: 0
Payload:

deviceType: Robot Vacuum Cleaner S10
deviceId: XXXXXXX

Received 09:26:13 => POLLING

QoS: 0
Payload:

deviceId: XXXXXXX
deviceType: WoSweeperOrigin
workingStatus: Clearing
onlineStatus: online
battery: 99
waterBaseBattery: 100
taskType: cleanAll

`

Corresponding Logs:

08:31:27 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Send GET,devices/XXXXXXX/status,(null)
08:31:29 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Response GET,https://api.switch-bot.com/v1.1/devices/XXXXXXX/status,{"statusCode":100,"body":{"deviceId":"XXXXXXX","deviceType":"WoSweeperOrigin","workingStatus":"ChargeDone","onlineStatus":"online","battery":100,"waterBaseBattery":100,"taskType":"standBy"},"message":"success"}
09:24:07 trce: SwitchBotMqttApp.Services.MqttCoreService[0] receive command XXXXXXX,{   "CommandType": "command",   "Command": "startClean",   "ParamName": "times",   "ParamValue": "2" }
09:24:10 trce: SwitchBotMqttApp.Services.MqttCoreService[0] receive command XXXXXXX,{   "CommandType": "command",   "Command": "startClean",   "ParamName": "action",   "ParamValue": "sweep_mop" }
09:24:12 trce: SwitchBotMqttApp.Services.MqttCoreService[0] receive command XXXXXXX,{   "CommandType": "command",   "Command": "startClean",   "ParamName": "fanLevel",   "ParamValue": "3" }
09:24:18 trce: SwitchBotMqttApp.Services.MqttCoreService[0] receive command XXXXXXX,{   "CommandType": "command",   "Command": "startClean",   "ParamName": "btn",   "ParamValue": "action" }
09:24:18 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Send POST,devices/XXXXXXX/commands,{"commandType":"command","command":"startClean","parameter":{"action":"sweep_mop","param":{"fanLevel":3,"waterLevel":2,"times":2}}}
09:24:20 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Response POST,https://api.switch-bot.com/v1.1/devices/XXXXXXX/commands,{"statusCode":100,"body":{},"message":"success"}
09:24:25 info: SwitchBotMqttApp.Controllers.WebhookController[0] {"eventType":"changeReport","eventVersion":"1","context":{"deviceType":"Robot Vacuum Cleaner S10","deviceMac":"XXXXXXX","timeOfSample":1725002665366},"onlineStatus":"online","taskType":"cleanAll"}
09:24:33 info: SwitchBotMqttApp.Controllers.WebhookController[0] {"eventType":"changeReport","eventVersion":"1","context":{"deviceType":"Robot Vacuum Cleaner S10","deviceMac":"XXXXXXX","timeOfSample":1725002673422},"onlineStatus":"online","workingStatus":"Clearing"}
09:24:35 info: SwitchBotMqttApp.Controllers.WebhookController[0] {"eventType":"changeReport","eventVersion":"1","context":{"deviceType":"Robot Vacuum Cleaner S10","deviceMac":"XXXXXXX","timeOfSample":1725002675762},"onlineStatus":"online","workingStatus":"Clearing"}
09:24:45 info: SwitchBotMqttApp.Controllers.WebhookController[0] {"eventType":"changeReport","eventVersion":"1","context":{"deviceType":"Robot Vacuum Cleaner S10","deviceMac":"XXXXXXX","timeOfSample":1725002685412},"onlineStatus":"online","workingStatus":"Clearing"}
09:25:35 info: SwitchBotMqttApp.Controllers.WebhookController[0] {"eventType":"changeReport","eventVersion":"1","context":{"deviceType":"Robot Vacuum Cleaner S10","deviceMac":"XXXXXXX","timeOfSample":1725002735455},"onlineStatus":"online","battery":99}
09:26:11 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Send GET,devices/XXXXXXX/status,(null)
09:26:13 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Response GET,https://api.switch-bot.com/v1.1/devices/XXXXXXX/status,{"statusCode":100,"body":{"deviceId":"XXXXXXX","deviceType":"WoSweeperOrigin","workingStatus":"Clearing","onlineStatus":"online","battery":99,"waterBaseBattery":100,"taskType":"cleanAll"},"message":"success"}

hsakoh commented 2 weeks ago

Thank you for the feedback.

In #27, the webhook wasn't being sent, but it seems that it is now being delivered.

However, the webhook being sent appears to have a structure different from what is described in the OpenAPI specification... According to the specification: inside context, but in reality: outside context.

expect:

{
    "eventType": "changeReport",
    "eventVersion": "1",
    "context": {
        "deviceType": "Robot Vacuum Cleaner S10",
        "deviceMac": DEVICE_MAC_ADDR,
        "workingStatus"："StandBy",
        "onlineStatus": "online",
        "battery": 100,// 0-100
        "waterBaseBattery": 100,
        "taskType": "explore",
        "timeOfSample": 123456789
    }
}

actual:

{
  "eventType": "changeReport",
  "eventVersion": "1",
  "context": {
    "deviceType": "Robot Vacuum Cleaner S10",
    "deviceMac": "XXXXXXX",
    "timeOfSample": 1725002665366
  },
  "onlineStatus": "online",
  "taskType": "cleanAll"
}

If it’s just a matter of changing the reading position, it would be simple, but there are other issues. It seems that only a portion of the state is received with a single webhook.

In the current add-on, state topics are divided into three categories: those that can be obtained only through the API, those that can be obtained only through the webhook, and those that can be obtained through both. Therefore, if we accept a webhook with missing parts, the values of other sensors will become unknown.

I need to either modify the status topics to be divided by sensor or wait until the webhook is sent according to the specification, but I would like to take a little more time to assess this.

First, I will proceed with the following tasks. Please wait for a while.

Provide feedback on https://github.com/OpenWonderLabs/SwitchBotAPI/issues/321 that the webhook is now being sent, but the payload differs from the specification.
Re-verify whether, when the missing payload is sent to the status topic, the previous state of the missing sensor is overwritten. (It should have been the case when I tested it before, but I will double-check.)

hsakoh commented 2 weeks ago

After rechecking, even when the payload in the status topic was missing, the values did not become unknown. (It seems I was mistaken.)

I'm not sure which version of the OpenAPI specification is correct, but since I don't expect much of a response from the issue on the SwitchBot, I plan to modify the implementation to "retrieve the value from inside context if it's there, and from the outside if it's not."

With the API specifications for K10 and K10Pro now available, I intend to release the update, likely by the end of today(in JST few hours). Once it's released, I will add a comment here.

hsakoh commented 2 weeks ago

I have just fixed it and released version v1.0.24. I would appreciate your feedback after trying it out.

hsakoh commented 2 weeks ago

I have one question I'd like to ask. I want to know what possible values deviceType can take. Specifically, I want to know the value of the S10 in the DeviceList API (when you press the [Fetch Devices By SwitchBotApi] button on the Ingress page).

The webhook shows Robot Vacuum Cleaner S10, while the Status API shows WoSweeperOrigin, which is quite different from the specification. So, I'd like to understand the value in the DeviceList as well.

sVnsation commented 2 weeks ago

I will do some further testing later. But i can say taskType and workingStatus are set now. But sometimes one of the both values is empty.

Here is the result of the DeviceList API:


14:43:44 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Send GET,devices,(null)
14:43:45 trce: SwitchBotMqttApp.Logics.SwitchBotApiClient[0] Response GET,https://api.switch-bot.com/v1.1/devices,
{
    "body": {
        "deviceList": [
            {
                "deviceId": "XXXXXXXXX",
                "deviceName": "Floor Cleaning Robot S10 1E",
                "deviceType": "Robot Vacuum Cleaner S10",
                "enableCloudService": true,
                "hubDeviceId": ""
            }
        ],
        "infraredRemoteList": [
        ]
    },
    "message": "success",
    "statusCode": 100
}

hsakoh commented 2 weeks ago

Thank you for the device list values.

Additionally, I checked the behavior of the MQTT integration. For numeric fields (sensors) like battery or temperature, if they are not present in the payload, their values remain unchanged. However, for string-based fields (sensors), they seem to be converted to empty strings.

The previous check was conducted only on numeric fields.

Please give me a more time. I will investigate whether the issue can be avoided with the MQTT Discovery sensor configuration. If that doesn't work, we can either break down the topics into more detailed fields or wait for the webhook specification to be corrected.

hsakoh commented 2 weeks ago

I have just fixed it and released version v1.0.25.

As a specific change, I have modified the value_template of the MQTT sensor. he new template now checks for a value, and if it exists, it uses that value; otherwise, it falls back to the current value. This change prevents the value from becoming "unknown" even if part of the Webhook data is missing.

"value_template": "{{ value_json['battery'] if (value_json['battery'] is defined and value_json['battery'] is not none) else states('sensor.both_battery_FFF921xxxxxx') }}"

I would appreciate your feedback after trying it out.

sVnsation commented 2 weeks ago

This works fine. Nice work👍

I just have one little inconsistency: On polling also the webhook_timestamp is updated.

hsakoh commented 2 weeks ago

Thank you for confirming !

Regarding the timestamp issue, it was an oversight during the fix. The state topic payload was referencing the same value.

I feel like the add-on is being updated too frequently, so I'll address this in the next update when I make other changes.

hsakoh / switchbot-mqtt

Device S10 Status is not send via MQTT when received via webhook #40