Single payload numbers expected as strings not processed properly.

[thomasleitner]

Hi,

First of all, thanks for providing this piece of software. I use it to connect an AiiBot Air Purifier (model A500) to my FHEM home control system via MQTT.

Now I have a strange problem: I can turn the Purifier On and Off, I can change the mode from Auto to Manual etc. all without problems. However: I cannot change the speed, even though I use exactly the same mechanisms.

Via MQTT, I try to

$ mosquitto_pub -t tuya/aiibot_air_purifier/dps/4/command -m \"1\"

BTW: I need the ", otherwise, it would not react at all (don't hear a beep from the device).

Now the strange thing I found out is: The only way to properly control the speed is to use the "--raw-value" option for tuya-cli. So this works:

fhem@ha:~$ tuya-cli set --ip 10.10.1.42 --id xxxxxxxxxxxxx --key yyyyyyyyyyy --protocol-version 3.3 --dps 4 --set 4 --raw-value
Set succeeded.

while this does not work and tuya-cli even throws an error:

fhem@ha:~$ tuya-cli set --ip 10.10.1.42 --id xxxxxxxxxxxxxxx --key yyyyyyyyyyyyy --protocol-version 3.3 --dps 4 --set 4
events.js:170
      throw er; // Unhandled 'error' event
      ^

Error: Error from socket
    at Socket.client.on.err (/usr/lib/node_modules/@tuyapi/cli/node_modules/tuyapi/index.js:351:30)
    at Socket.emit (events.js:193:13)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at processTicksAndRejections (internal/process/task_queues.js:81:17)
Emitted 'error' event at:
    at Socket.client.on.err (/usr/lib/node_modules/@tuyapi/cli/node_modules/tuyapi/index.js:351:16)
    at Socket.emit (events.js:193:13)
    [... lines matching original stack trace ...]
    at processTicksAndRejections (internal/process/task_queues.js:81:17)

Ok, I know that this sounds more like a tuya-api problem, but how can --raw-value be supported by tuya-mqtt?? What's the difference anyway?

Thanks // Tom

BTW: The --raw-value option apparently avoids that someone messes with the quotes. See here: codetheweb/tuyapi#254 ...

PS .. this is my devices.conf:

[
  {
    name: 'Aiibot Air Purifier',
    id: 'xxxxxxxxx',
    key: 'yyyyyyyy',
    ip: '10.10.1.42',
    version: '3.3',
    type: 'GenericDevice',
        template: {
            power: {
              key: '1',
              type: 'bool'
            },
            pm25: {
              key: '2',
              type: 'int'
            },
            mode: {
              key: '3',
              type: 'string'
            },
            speed: {
              key: '4',
              type: 'string'
            }
        }
  }
]

[thomasleitner]

I was digging more into the problem now and found:

$ mosquitto_pub -t tuya/aiibot_air_purifier/dps/4/command -m \"2\"

results in:

2020-10-26T12:03:29.929Z mqttjs:client _handlePublish: packet Packet { cmd: 'publish', retain: false, qos: 0, dup: false, length: 43, topic: 'tuya/aiibot_air_purifier/dps/4/command', payload: <Buffer 22 32 22> }
2020-10-26T12:03:29.929Z mqttjs:client _handlePublish: qos 0
2020-10-26T12:03:29.929Z tuya-mqtt:command Received MQTT message ->  {"topic":"tuya/aiibot_air_purifier/dps/4/command","message":"\"2\""}
2020-10-26T12:03:29.930Z tuya-mqtt:command Received command for DPS4:  "2"
2020-10-26T12:03:29.931Z tuya-mqtt:tuyapi Set device 00420238f4cfa24ee030 -> {"dps":"4","set":"\"2\""}
2020-10-26T12:03:29.932Z TuyAPI SET Payload:
2020-10-26T12:03:29.932Z TuyAPI { devId: '00420238f4cfa24ee030',
  gwId: '00420238f4cfa24ee030',
  uid: '',
  t: 1603713809,
  dps: { '4': '"2"' } }

which is wrong because the payload of '"2"' has extra quotes and my device would not accept it.

But when I use:

$ mosquitto_pub -t tuya/aiibot_air_purifier/dps/4/command -m 2

I get:

2020-10-26T12:12:45.692Z mqttjs:client _handlePublish: packet Packet { cmd: 'publish', retain: false, qos: 0, dup: false, length: 41, topic: 'tuya/aiibot_air_purifier/dps/4/command', payload: <Buffer 32> }
2020-10-26T12:12:45.692Z mqttjs:client _handlePublish: qos 0
2020-10-26T12:12:45.692Z tuya-mqtt:command Received MQTT message ->  {"topic":"tuya/aiibot_air_purifier/dps/4/command","message":"2"}
2020-10-26T12:12:45.693Z tuya-mqtt:command Received command for DPS4:  2
2020-10-26T12:12:45.693Z tuya-mqtt:tuyapi Set device 00420238f4cfa24ee030 -> {"dps":"4","set":2}
2020-10-26T12:12:45.694Z TuyAPI SET Payload:
2020-10-26T12:12:45.694Z TuyAPI { devId: '00420238f4cfa24ee030',
  gwId: '00420238f4cfa24ee030',
  uid: '',
  t: 1603714365,
  dps: { '4': 2 } }

Which is also not accepted by my device because 2 is not a string!

So it seems as if tuya-mqtt has the same problem as tuya-cli ... I tuya-cli this was solved with the --raw-value flag.

[Flip76]

Did you try to send the value without masking the doublequotes?

mosquitto_pub -t tuya/aiibot_air_purifier/dps/4/command -m "2"

[thomasleitner]

Did you try to send the value without masking the doublequotes?

mosquitto_pub -t tuya/aiibot_air_purifier/dps/4/command -m "2"

Thank you, but passing -m "2" is the same as passing -m 2 .... The quotes are removed by the shell.

I've tried it again, and the result was as expected:

2020-10-26T12:55:51.402Z tuya-mqtt:command Received MQTT message ->  {"topic":"tuya/aiibot_air_purifier/dps/4/command","message":"2"}
2020-10-26T12:55:51.403Z tuya-mqtt:command Received command for DPS4:  2
2020-10-26T12:55:51.403Z tuya-mqtt:tuyapi Set device 00420238f4cfa24ee030 -> {"dps":"4","set":2}
2020-10-26T12:55:51.404Z TuyAPI SET Payload:
2020-10-26T12:55:51.404Z TuyAPI { devId: '00420238f4cfa24ee030',
  gwId: '00420238f4cfa24ee030',
  uid: '',
  t: 1603716951,
  dps: { '4': 2 } }

Which is again not what I need. I would need this:

dps: { '4': "2" } }

and I cannot get this from tuya-mqtt now ...

[Flip76]

Ok, I see... I guess the easiest way would be to add an option like:

tuya//dps/<dps#>/command/raw

@tsightler What do you think?

[tsightler]

The core issue here is that MQTT only supports simple string or JSON string messages while tuya protocol uses JSON which allows string or number types. Basically, for the DPS keys, everything received via MQTT is a string, and the code simply makes an assumption that, if it's a number, it should be sent via JSON as a number type, not a string. This is correct for most use cases, but it appears that your device actually expects a number for speed, but expects it to be sent as a string.

There are two ways to deal with this, one is the using the DPS/command topic and just send the raw JSON exactly as you want it to be sent to the tuya device. Just command topic expects raw Tuya JSON data so you can literally just send {"dps":"4","set":"2"} to that topic and it will forward that directly to the device.

The other option is to actually fix your template and use it. Right now you are defining a template, but it's using a slightly incorrect format, instead you should use something like this:

[
  {
    name: 'Aiibot Air Purifier',
    id: 'xxxxxxxxx',
    key: 'yyyyyyyy',
    ip: '10.10.1.42',
    version: '3.3',
    type: 'GenericDevice',
        template: {
            power_state: {
              key: '1',
              type: 'bool'
            },
            pm25_state: {
              key: '2',
              type: 'int'
            },
            mode_state: {
              key: '3',
              type: 'string'
            },
            speed_state: {
              key: '4',
              type: 'string'
            }
        }
  }
]

Then you would be able to get the state of the device via the template defined state topics:

tuya/aiibot_air_purifier/power_state
tuya/aiibot_air_purifier/pm25_state
tuya/aiibot_air_purifier/mode_state
tuya/aiibot_air_purifier/speed_state

This power_state would show "ON/OFF" instead of "0/1" while the other states should show the same values, however, the critical piece is that you could then control the speed using the automatically created speed_command topic:

tuya/aiibot_air_purifier/speed_command

Because the template explicitly says that the DPS 4 value is a string it will simple send the string value to the DPS 4 topic, rather than trying to process it into a number. That's the entire point of the template system, not just to make the topics shorter and easier use, but to allow providing the needed contextual information about the topics so that tuya-mqtt has the data needed to process the incoming string data rather than having to guess based solely on the content.

That being said, perhaps a "raw_command" option for the DPS topics wouldn't be a bad idea, but you should be able to use the existing capabilities to do what you need.

[thomasleitner]

@tsightler : Thanks a lot for your detailed reply. This works. Looks as I didn't read the documentation for devices.conf thoroughly enough.

In fact I also tried sending JSON payloads directly, but it always complained about invalid Tuya JSON payloads, so I gave up.

Now, I'm happily using the devices.conf you've corrected above and I use tuya/aiibot_air_purifier/speed_command to control the speed.

So finally, a raw_command is not necessary for me anymore and with all the capabilities you mention above, I doubt that it's worthwhile implementing it.

Thanks a lot // Tom

[tsightler]

Thanks for confirming that the template system worked as expected. It's one thing to design it and another thing entirely to see it actually work for such a use case so I'm really glad to hear this.

That being said, your post has definitely caused me to think about a better way to deal with the raw DPS keys in the future. I imagine there are very few cases where Tuya device would expect a quoted string within the JSON so my current thought is that I'll modify the handling of the DPS key command topics to recognize an MQTT message containing a quoted string and treat that case as a literal value, sending it exactly as it was received regardless if it was a number or not. That way at least there is a method to send number values as strings and you're second method above would have worked.

Obviously the ideal case is still to use the template, but this would make the DPS key topics a little more flexible and make it easier to test with as well so I really appreciate you brining up this case and providing such detailed information about it. One of the challenges with developing something like this is that there are so many diverse Tuya devices it's very difficult to imagine all of the possible variations and build something generic enough to address them but simple enough to be usable.

[thomasleitner]

Thanks again for your reply. Normally, with a 25+ years experience in IT and Linux, I don't have many problems installing and configuring software. However, here it looks as there are still some hidden pitfalls. So I think improving it as you describe might definately be beneficial for other users.

While experimenting with the software, I found some other issues which I'll post separately -- just to let you know.

Thanks again for your efforts!