TUTORIAL: Get all hidden DP's , based on api from Android App (tuya or smartlife)

[pergolafabio]

Hi, yes i'm aware of this one: https://github.com/rospogrigio/localtuya/wiki/HOWTO-get-a-DPs-dump But for a lot of devices it doesnt provide the full dp's list, it sometimes even gives errors...

Also iot.tuya.com, not all DP's are visible, for some reason lots of DP's are just hidden...

I have found a way, based on reverse engeneering with the android app using this one: https://github.com/TuyaAPI/cloud

Install it with this command: npm i @tuyapi/cloud

Create a file keys.json, depending if you use smartlife or tuya , paste the content below:

For Tuya:

{
  "key": "3fjrekuxank9eaej3gcx",
  "secret": "aq7xvqcyqcnegvew793pqjmhv77rneqc",
  "secret2": "vay9g59g9g99qf3rtqptmc3emhkanwkx",
  "certSign": "93:21:9F:C2:73:E2:20:0F:4A:DE:E5:F7:19:1D:C6:56:BA:2A:2D:7B:2F:F5:D2:4C:D5:5C:4B:61:55:00:1E:40" 
}

For Smartlife:

{
  "key": "ekmnwp9f5pnh3trdtpgy",
  "secret": "r3me7ghmxjevrvnpemwmhw3fxtacphyg",
  "secret2": "jfg5rs5kkmrj5mxahugvucrsvw43t48x",
  "certSign": "0F:C3:61:99:9C:C0:C3:5B:A8:AC:A5:7D:AA:55:93:A2:0C:F5:57:27:70:2E:A8:5A:D7:B3:22:89:49:F8:88:FE" 
}

Create this file "cloud.js" , and fill in your email/password

const apiKeys = require('./keys.json');
const Cloud = require('@tuyapi/cloud');
const is = require('is');
const api = new Cloud({key: apiKeys.key, secret: apiKeys.secret,secret2: apiKeys.secret2, certSign: apiKeys.certSign, apiEtVersion: '0.0.1', region: 'EU'});
api.loginEx({email: "XXXX", password: "XXXX"}).then(async sid => {
  console.log(sid);

  api.request({action: 'tuya.m.location.list'}).then(async groups => {
    for (const group of groups) {
      api.request({action: 'tuya.m.my.group.device.list', gid: group.groupId}).then(async devicesArr => {
        for (const device of devicesArr) {
           console.log('group: "%s"\tdevice: "%s"\tdevDPS: "%s"', group.name, device.name, device.dps);
        }
      });
    }
  });
});

Run it
node cloud.js

It gives outputs like below... For me the 201 untill 208 were just not visible with debug or iot, and those ones were the most important ones for me :-) I hope this can be helpfull for others...

Maybe usefull for somekind of wiki @rospogrigio ?

[osnwt]

Thank you for very useful info.

[rospogrigio]

Well this is shocking. Unfortunately the only device I own that sometimes does not return DP 1 data is now working properly and I don't know how to trigger that problem. @osnwt , does this work with your device that was hiding some DP? Don't know whether this is applicable to the local calls, however, need to investigate more. But this is super precious information, thank you!

[pergolafabio]

You are welcome, hopefully I can be useful to make this local stuff even better

[osnwt]

Yes, this way I've got few more DP comparing to other method. BTW, using this code, Tuya reports: "Android phone is connected using your account", just FYI.

Attached is a side by side comparison of DP lists received using different methods. I had another device (currently not connected) where some DP listed via API explorer are of type "json", so I even asked you what type of DP I should choose to describe such (new for new protocol?) type of DP...

It also seems that this is not related to missing detection of some DP because in this case both methods returned DP 113 when the latest localtuya did not show it to me after delete/add... When device was first added, I had all true/false type DP for switches. After I removed it and tried to add again I had only 4 DP shown (101, 109, 110, 111).

[pergolafabio]

Yes, this way I've got few more DP comparing to other method. BTW, using this code, Tuya reports: "Android phone is connected using your account", just FYI.

yeah, i have that too, its normal since you are simulating a login based on an android app

[osnwt]

Here is an API explorer result for "Get the specifications and properties of the device" for that offline device (a kind of Fairy lights where you can program visual scenes and patterns via Tuya app). You see some kinds of DP have type "json". So maybe the base64 strings above are also DP with JSON type, encoded, for instance, as BSON+base64. But for this exact device API explorer does not return such specification...

{
  "result": {
    "category": "dd",
    "functions": [
      {
        "code": "switch_led",
        "desc": "{}",
        "name": "开关",
        "type": "Boolean",
        "values": "{}"
      },
      {
        "code": "work_mode",
        "desc": "{\"range\":[\"white\",\"colour\"]}",
        "name": "模式",
        "type": "Enum",
        "values": "{\"range\":[\"white\",\"colour\"]}"
      },
      {
        "code": "bright_value",
        "desc": "{\"min\":10,\"max\":1000,\"scale\":0,\"step\":1}",
        "name": "亮度值",
        "type": "Integer",
        "values": "{\"min\":10,\"max\":1000,\"scale\":0,\"step\":1}"
      },
      {
        "code": "temp_value",
        "desc": "{\"min\":0,\"max\":1000,\"scale\":0,\"step\":1}",
        "name": "冷暖值",
        "type": "Integer",
        "values": "{\"min\":0,\"max\":1000,\"scale\":0,\"step\":1}"
      },
      {
        "code": "colour_data",
        "desc": "{\"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}}",
        "name": "彩光",
        "type": "Json",
        "values": "{\"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}}"
      },
      {
        "code": "scene_data",
        "desc": "{\"scene_num\":{\"min\":1,\"scale\":0,\"max\":8,\"step\":1},\"scene_units\": {\"unit_change_mode\":{\"range\":[\"static\",\"jump\",\"gradient\"]},\"unit_switch_duration\":{\"min\":0,\"scale\":0,\"max\":100,\"step\":1},\"unit_gradient_duration\":{\"min\":0,\"scale\":0,\"max\":100,\"step\":1},\"bright\":{\"min\":0,\"scale\":0,\"max\":1000,\"step\":1},\"temperature\":{\"min\":0,\"scale\":0,\"max\":1000,\"step\":1},\"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}}}",
        "name": "场景",
        "type": "Json",
        "values": "{\"scene_num\":{\"min\":1,\"scale\":0,\"max\":8,\"step\":1},\"scene_units\": {\"unit_change_mode\":{\"range\":[\"static\",\"jump\",\"gradient\"]},\"unit_switch_duration\":{\"min\":0,\"scale\":0,\"max\":100,\"step\":1},\"unit_gradient_duration\":{\"min\":0,\"scale\":0,\"max\":100,\"step\":1},\"bright\":{\"min\":0,\"scale\":0,\"max\":1000,\"step\":1},\"temperature\":{\"min\":0,\"scale\":0,\"max\":1000,\"step\":1},\"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}}}"
      },
      {
        "code": "countdown",
        "desc": "{\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}",
        "name": "倒计时剩余时间",
        "type": "Integer",
        "values": "{\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}"
      },
      {
        "code": "control_data",
        "desc": "{\"change_mode\":{\"range\":[\"direct\",\"gradient\"]}, \"bright\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}, \"temperature\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}, \"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":255,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":255,\"step\":1}}",
        "name": "调节",
        "type": "Json",
        "values": "{\"change_mode\":{\"range\":[\"direct\",\"gradient\"]}, \"bright\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}, \"temperature\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}, \"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":255,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":255,\"step\":1}}"
      }
    ],
    "status": [
      {
        "code": "switch_led",
        "name": "开关",
        "type": "Boolean",
        "values": "{}"
      },
      {
        "code": "work_mode",
        "name": "模式",
        "type": "Enum",
        "values": "{\"range\":[\"white\",\"colour\"]}"
      },
      {
        "code": "bright_value",
        "name": "亮度值",
        "type": "Integer",
        "values": "{\"min\":10,\"max\":1000,\"scale\":0,\"step\":1}"
      },
      {
        "code": "temp_value",
        "name": "冷暖值",
        "type": "Integer",
        "values": "{\"min\":0,\"max\":1000,\"scale\":0,\"step\":1}"
      },
      {
        "code": "colour_data",
        "name": "彩光",
        "type": "Json",
        "values": "{\"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}}"
      },
      {
        "code": "scene_data",
        "name": "场景",
        "type": "Json",
        "values": "{\"scene_num\":{\"min\":1,\"scale\":0,\"max\":8,\"step\":1},\"scene_units\": {\"unit_change_mode\":{\"range\":[\"static\",\"jump\",\"gradient\"]},\"unit_switch_duration\":{\"min\":0,\"scale\":0,\"max\":100,\"step\":1},\"unit_gradient_duration\":{\"min\":0,\"scale\":0,\"max\":100,\"step\":1},\"bright\":{\"min\":0,\"scale\":0,\"max\":1000,\"step\":1},\"temperature\":{\"min\":0,\"scale\":0,\"max\":1000,\"step\":1},\"h\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":360,\"step\":1},\"s\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1},\"v\":{\"min\":0,\"scale\":0,\"unit\":\"\",\"max\":1000,\"step\":1}}}"
      },
      {
        "code": "countdown",
        "name": "倒计时剩余时间",
        "type": "Integer",
        "values": "{\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}"
      }
    ]
  },
  "success": true,
  "t": 1675082260857,
  "tid": "e2ba87e8a09a11ed97b7f276006a160b"
}

[rospogrigio]

My impression is that what you put in the keys.json is a developer certificate that allows to have access to debug/development DPs, do you agree?

[osnwt]

As far as I see, they were extracted from reverse engineered Android app. So they are, rather, internal certs for the app, not a dev ones.

In case of my device we see that all missing DP are base64-encoded, and decoding does not return a text result. From another point, how to present json DP shown in the API explorer spec? They are neither number, nor boolean or string. It is a 3.4 device, BTW.

[pergolafabio]

yes, indeed, i extracted them from the android apk app

[rospogrigio]

In case of my device we see that all missing DP are base64-encoded, and decoding does not return a text result

Maybe it's binary? Would it make sense?

[osnwt]

For reference:

DP 114: AwMDAw==
03 03 03 03

DP 122: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAtAoAAAEAAAAACgAAAAAAAAAAAAAAAAAAAAAAAA==
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 b4 0a 00 00 01 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

DP 145: Bf8E/w==
05 ff 04 ff

Looks like an arbitrary binary data.

[rospogrigio]

You should check if they change over time, and how, and if they are related to other DPs in any way. For example, 114 contains four 3, which is the value of DP 116, perhaps it's a sort of history of that DP? I have noticed that everything in the Tuya world has its sense so those value must mean something.

[osnwt]

What I see is that Tuya doesn't enforce API declarations for API explorer (again, just guessing). I see many devices either don't have any API spec shown, or don't show full DP lists. For example, the Smart Light device for which I posted an API Explorer spec, returns this list of DP. Even number of them is different comparing to the spec...

device: "Smart Light"
devDPS: "{
  '20': false,
  '21': 'pre_set',
  '22': 1000,
  '23': 500,
  '24': '010e03e801f0',
  '25': '01f800020',
  '26': 0,
  '28': '',
  '101': '0C0C0C07',
  '102': 0,
  '103': 200,
  '104': 1,
  '105': ''
}"

[pergolafabio]

wll, in my case, all the hidden values are still the same, as in the first post of the thread, the ones starting from 200 are still the same i use them to feed my cats :-)

[osnwt]

You should check if they change over time, and how, and if they are related to other DPs in any way

Definitely they mean something. In my case I am not too much interested in decoding all details because the device is modular and I don't have all possible modules installed. Tuya app shows me some stats about water counters, and event logs. But I only need a bitmap of water leak sensors connected, and I will watch for their changes.

But an ability to see all data the App has is priceless. I bought a Tuya ZigBee hub just to see if my switches have some more settings comparing to already supported by zigbee2mqtt. Now I can see them directly.

[uzlonewolf]

Also iot.tuya.com, not all DP's are visible, for some reason lots of DP's are just hidden...

Actually you can tell it to give you all DPs by changing the Instruction mode from "Standard Instruction" to "DP Instruction" https://github.com/jasonacox/tinytuya/discussions/284#discussioncomment-4953888

[pergolafabio]

Hi , yes, but that's not working for all device types

[uzlonewolf]

but that's not working for all device types

What device types does it not work for? It's worked for every device I've tried, though it usually takes 12-24 hours for the change to propagate.

Anyway, there is actually a Python version of this for those who don't want to install Node.js: https://github.com/rospogrigio/localtuya/pull/1131 and a stand-alone version https://gist.github.com/avataar/2a6ee4f58aaedfcc062a838380f3cffb (note: to get the DPS list you'll need to change line 185 from devs[dev["devId"]] = self._map_device(dev) to just devs[dev["devId"]] = dev). For these, the format of the secret is '_'.join( (certSign, secret2, secret) )

[pergolafabio]

i have a smart pet feeder, i only saw listed from 100 to 108 , the ones above 200 were not visible

[pergolafabio]

ah, maybe you are right, seems they are indeed more DP listed there, thnx!

[uzlonewolf]

To add to the keys.json collection:

For the Ledvance app:

{
  "key": "fx3fvkvusmw45d7jn8xh",
  "secret": "cgqx3ku34mh5qdesd7fcaru3gx7tyurr",
  "secret2": "armptsqyfpxa4ftvtc739ardncett3uy",
  "certSign": "A" 
}

For the Sylvania Smart WiFi app:

{
  "key": "creq75hn4vdg5qvrgryp",
  "secret": "wparh3scdv8dc7rrnuegaf9mqmn4snpk",
  "secret2": "ag4xcmp9rjttkj9yf9e8c3wfxry7yr44",
  "certSign": "A" 
}

[szupi-ipuzs]

BirdLover app:

{
  "key": "gmusrthh3sygeyv3sr38",
  "secret": "x4y4ds9nysv4d3agjyqwmvnptwhgtcwu",
  "secret2": "pku4cchspfmskfgtaacqcvkfdscx7u7t",
  "certSign": "A"
} 

[szupi-ipuzs]

I believe these to be Gosund app secrets:

{
  "key": "pwhnn4fa7ydkakf3nehn",
  "secret": "pqdyxyx3uhk337sxxumdgfry3awaxysm",
  "secret2": "wm8hvxahqhcvvnpqgurympm4ppfgxxnm",
  "certSign": "A"
} 

[szupi-ipuzs]

Brennenstuhl Connect:

{
  "key": "dh35afm9ha79sppyxgkf",
  "secret": "aqy9p3e78xr5htpsn95fss5rvcdtaahd",
  "secret2": "9gyrek4h5ygwshsndqurwjkddtjpw9yr",
  "certSign": "A"
}

[pergolafabio]

Thnx for sharing

[AmirKeshavarz]

{
  "result": {
    "category": "dd",
    "functions": [
      {
        "code": "switch_led",
        "desc": "{}",
        "name": "开关",
        "type": "Boolean",
        "values": "{}"
      },
      {
        "code": "countdown",
        "name": "倒计时剩余时间",
        "type": "Integer",
        "values": "{\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}"
      }
    ]
  },
  "success": true,
  "t": 1675082260857,
  "tid": "e2ba87e8a09a11ed97b7f276006a160b"
}

Hi, I'm wondering about "code" field here! "name" is clearly useful when Tuya shows a dp to a user. Also dp-id which is not visible in this device spec is useful for messaging between cloud and device.

But what is the role of "code" in tuya systems?