New Cloud functions to download DP Name mappings

[uzlonewolf]

Adds mappings.json to hold the DP<->Name mappings and various functions to download, save, and load it. mappings.json contains a dict with product IDs as keys.

Submitting as a draft as I still need to add documentation but wanted to show the direction I was going in after the discussion in #353.

[mschlenstedt]

So if I understood your changes correctly, the wizard will create the mappings.json? So I do not have to handle this in PF #353 ? I just call "getmappings( self, devices )" and that's it?

[mschlenstedt]

Ah, no, I have to call "load_mappings(mappings)", right?

[uzlonewolf]

If you only have 1 or 2 devices then you can use the new find_dp_mapping(product_id or device_id) function I just added. Otherwise, if you have more, I recommend something along the lines of:

import tinytuya
import json

mappings = tinytuya.load_dp_mappings()

with open(tinytuya.DEVICEFILE, "r") as infile:
    tuyadevices = json.load( infile )

for dev in tuyadevices:
    try:
        devid = dev['id']
        productid = dev['product_id']
    except:
        # we need both the device id and the product id to download mappings!
        print( 'No device id and/or product id!', dev )
        continue

    if productid in mappings:
        print( 'Mapping for device %s:' % devid, mappings[productid] )
    else:
        print( 'Device %s has no mappings!' % devid )

[uzlonewolf]

With that, I think this first half is done. I have the "download mappings?" question in the wizard defaulting to "no" for now, not sure if I should change that to "yes"

[uzlonewolf]

Ok, I added another function to assign mappings to an entire tuyadevice list. Now all you need to do is:

import tinytuya
import json

with open(tinytuya.DEVICEFILE, "r") as infile:
    tuyadevices = json.load( infile )

print( 'before:', tuyadevices )
tinytuya.assign_dp_mappings( tuyadevices )
print( 'after:', tuyadevices )

or

import tinytuya
import json

with open(tinytuya.DEVICEFILE, "r") as infile:
    tuyadevices = json.load( infile )

mappings = { ... }

print( 'before:', tuyadevices )
tinytuya.assign_dp_mappings( tuyadevices, mappings )
print( 'after:', tuyadevices )

[mschlenstedt]

I have the "download mappings?" question in the wizard defaulting to "no" for now, not sure if I should change that to "yes"

I would change it to yes. Not much overload.

[uzlonewolf]

After ripping out DPMAPPINGSFILE and merging everything into devices.json, my devices.json went from 64 KiB / 1851 lines to 454 KiB / 13871 lines. I do have a ton of devices though (127 to be exact) so this may be acceptable, not sure. For comparison, my mappings.json was 115 KiB and 4935 lines.

[uzlonewolf]

Also, Cloud.getdevices() now skips downloading the factory-infos and DP mappings unless something (such as the local key) has changed. For my massive device list this drops the wizard run time from 34 seconds to ~5 in addition to saving multiple dozens of API calls.

[jasonacox]

Tested - no issues - released: https://pypi.org/project/tinytuya/1.12.8/

Thanks again @uzlonewolf - you're brilliant! 🙏

@mschlenstedt, the new devices.json should support the DP mappings for #353 now.

[mschlenstedt]

Yes, I will change my PR to use the new file. Will take a few days though.

If user requests to get DP mappings, and DEVICEFILE exists, prompt if they want to replace file or only add new devices

Haven't checked already, but I would prefer that the default ("yes") recreates the devices.json in total. I run the wizard from my Frontend via script this way:

yes | python3 -m tinytuya wizard

So everything what is not "yes" will be missed... An even better way of course would be to define the answers e.g. in ENVIRONMENT variables. But this is maybe another PR...

[jasonacox]

No rush! Thanks for your PR and engagement on this enhancement.

Haven't checked already, but I would prefer that the default ("yes") recreates the devices.json in total. I run the wizard from my Frontend via script this way:

yes | python3 -m tinytuya wizard

This works for me. The last commit @uzlonewolf made doesn't need to recreate devices.json, it enables the logic to add mappings to the device data if they are missing. I confirmed by restoring an old devices.json and running the wizard and saw that it added DP mappings to all devices.

[uzlonewolf]

An even better way of course would be to define the answers e.g. in ENVIRONMENT variables. But this is maybe another PR...

Actually this PR also abuses the scanner's -y/-yes flag to answer "yes" to the "Use existing credentials?" and "Download DP Name mappings?" questions but "no" to "Poll local devices?". I had added an argument to wizard() a long time ago to do this, and after the timing tests I did for this PR I just didn't remove the command line switch this time. Just call with python3 -m tinytuya wizard -yes. As a future PR it would probably be a good idea to change this to something more intuitive (perhaps -yes-cloud or something?) and add it to the help documentation.