Models data are not being fetched properly

[siddharthbiju]

Models data are not being fetched properly. '{"status":0,"encoding":"UTF-8","language":"zh_CN","data":[]}' are present in all model json files

@ysard

[ysard]

Yes you are right, there must be a modification of their API... Unfortunately I haven't the time to go deeper in it :/ You can still use the dump available in the release page of the project.

[Biswa96]

I can also reproduce this issue. Is it possible to use the db_dump option for downloading a specific device & brand combination? That will help to find things faster.

[Biswa96]

This issue happens because the "vendor" string is lost somewhere. The URL takes the default vendor string "mi" instead of the value from json. For example,

• Wrong:

https://sg-urc.io.mi.com/controller/code/1?country=FR&version=6034&ts=1697176154469&nonce=-404196826&matchid=kk_1_2083_2710&vendor=mi&opaque=4057109f64cc29204ac45c556d6ffc82c343acf7

• Right:

https://sg-urc.io.mi.com/controller/code/1?country=FR&version=6034&ts=1697176681257&nonce=412807635&matchid=kk_1_2083_2710&vendor=kk&opaque=3b38f672945496e20face7768b9c3cae7e65bb7e

See the "vendor" string difference - mi vs. kk.

[Biswa96]

Something happens in these lines https://github.com/ysard/mi_remote_database/blob/42e733ec20662f0d42e82357a89c62387e3fbd0a/src/xiaomi_query.py#L269-L282

I can not figure out how to solve it. Basically, load_ids_from_brands returns the vendor and model ids both but only model ids are passed to crawl_models.

[chip44]

based on the code before https://github.com/ysard/mi_remote_database/commit/97837a0582809c89b78e5306746b4f9f4a5a5b13, this can be fixed by changing line 282: https://github.com/ysard/mi_remote_database/blob/85e47f1dda81acf3ad3933e6e7ea8911cb6550a4/src/xiaomi_query.py#L281-L282 to this:

for vendor in ("mi", "kk", "mx", "xm", "yk"):
    crawl_models(models_path, model_ids, vendorid=vendor)

but now another problem is introduced, because the files might later get overwritten with empty data even if the result was correct. so a basic check can help avoid it with something like this:

if filepath.exists():
    with open(filepath, "r") as f:
        if len(json.load(f)["data"]) > 0:
            continue

instead of these lines: https://github.com/ysard/mi_remote_database/blob/85e47f1dda81acf3ad3933e6e7ea8911cb6550a4/src/xiaomi_query.py#L183-L184

feel free to correct me if im wrong, but it seems to work for me. i can also submit a pr if that helps :)

[ysard]

Thank you both for identifying the faulty lines. It was indeed a regression due to an unfortunate commit. This part of the code is not covered by unit-tests... That should be fixed now.