Rework Cloud device list fetching

[uzlonewolf]

Creating as a draft to get the conversation started.

Due to a number of reasons, I now have 2 App accounts: the one I use, and a 2nd I give out to people. The way the device list is currently fetched only returns the devices tied to the one account the provided Device ID is associated with. I went hunting and discovered https://developer.tuya.com/en/docs/cloud/fc19523d18?id=Kakr4p8nq5xsc which returns all devices for all accounts. The only downside is the list is stored in result->devices instead of result directly. It also needs to be looped if there are more than size devices, which is a problem I will soon run into as I'll be surpassing 100 devices later today or tomorrow (mainly thanks to $dayjob getting enamored with putting smart bulbs into track lights). On the upside it means people no longer need to find an initial Device ID when configuring tinytuya.

My main question is about handling the slightly different return format. I currently have it reformatting the list into the original format to keep from breaking anything which parses tuya-raw.json, but that causes it to drop some metadata and is no longer a true "this is exactly what we got from the server."

Old format:

{
    "result": [
        {
            "active_time": 1677110164,
            "biz_type": 18,
            "category": "tdq",
            "create_time": 1677110164,
...
            "update_time": 1652097381,
        }
    ],
    "success": true,
    "t": 1677263715070,
    "tid": "faxxxxxxxxxxxxxxxxxxxxxxee"
}

New format:

{
  "result": {
    "devices": [
      {
        "active_time": 1677110164,
        "biz_type": 18,
        "category": "tdq",
        "create_time": 1677110164,
...
    "update_time": 1652097381,
      }
    ],
    "has_more": false,
    "last_row_key": "08xxxxxxxxxxxxxxxxxxxxC8",
    "total": 98
  },
  "success": true,
  "t": 1677262499887,
  "tid": "26xxxxxxxxxxxxxxxxxxxxxxxxb2"
}

Should I write the new format to tuya-raw.json, or keep doing what I'm currently doing and reformatting it into the old format? With how I'm going to need to loop for multiple cloud calls, should I go with my initial plan of merging the multiple device lists into a single list or should I create a new list and store each call separately?

[jasonacox]

Brilliant discovery! One less step in wizard is a win!

I'm going to be a bit high maintenance and suggest that we keep the tuya-raw.json format the same, for the reason you already stated (others are using this). In hindsight, I should have used something like tuya-full.json which is more true to the intention of the file (providing more than what you get in devices.json). My thoughts:

• Keep the API contract and translate to stay true to the current format for others using this
• Combine all (e.g. >100) devices into this one json document
• Add a root element to help convey intention with versioning details - maybe something like:

{
    "file": {
        "name": "tuya-raw.json",
        "description": "Full raw list of Tuya devices.",
        "account": "xxxx",
        "date": "2023-02-25T09:24:17.453956",
        "tinytuya": "1.10.3"
    },
    "result": [{
        "active_time": 1673078929,
        "biz_type": 18,
        "category": "cz",

        "update_time": 1652097381
    }],
    "success": true,
    "t": 1677263715070,
    "tid": "faxxxxxxxxxxxxxxxxxxxxxxee"
}

Having the root element would convey that this is created/translated by tinytuya. We could also add if the file was "all devices for all accounts" or just one associated with a deviceid. I added an iso date in a readable format for the humans looking at the file to determine content (we have a timestamp t already).

[uzlonewolf]

tuya-raw.json:

{
    "result": [
        {
            "active_time": 1676171849,
            "biz_type": 18,
            "category": "cz",
            "create_time": 1676171849,
...
        }
    ],
    "success": true,
    "t": 1677884754157,
    "tid": "###############",
    "fetches": 3,
    "total": 109,
    "file": {
        "name": "tuya-raw.json",
        "description": "Full raw list of Tuya devices.",
        "account": "###########3",
        "date": "2023-03-03T15:22:04.059014",
        "tinytuya": "1.10.3"
    }
}

[jasonacox]

Looks good @uzlonewolf ! Thanks.

I'm going to run my basic tests and if all goes well, rev the version to 1.11.0. I debated if it needs a bump like that (vs. 1.10.4) but since we are adjusted tuya-raw.json, even if backward compatible (for JSON parsing), it might be good to signal a more significant change. Thoughts?

If it looks good, I'll release later today.

[uzlonewolf]

Sounds good to me. Not sure if we want to add some documentation; I kept going back-and-forth with myself and ended up making it so you can set c.use_old_device_list = True (along with including a Device ID) to use the old method if you need to for some reason.

I'm also working on some Contrib.IRDevice updates that I hope to get in soon; I have a bunch of other stuff to do today though so I don't know if I'll be able to get it done tonight.

[jasonacox]

Released v1.11.0. Thanks again, @uzlonewolf !

Good point on documentation. At least for the setup, I need to remove the instructions for DeviceID for wizard.