Request new Vacuum Xiaomi s12

alejarvis commented 1 year ago

Checklist

[X] I have updated the integration to the latest version available
[X] I have checked if the vacuum/platform is already requested
[ ] I have sent raw map file to piotr.machowski.dev [at] gmail.com (Retrieving map; please provide your GitHub username in the email)

What vacuum model do you want to be supported?

xiaomi.vacuum.b106eu

What is its name?

Xiaomi Robot Vacuum S12

Available APIs

[ ] xiaomi
[ ] viomi
[ ] roidmi
[ ] dreame

Errors shown in the HA logs (if applicable)

2023-08-19 05:08:13.598 ERROR (MainThread) [homeassistant.helpers.entity] Update for camera.xiaomi_cloud_map_extractor fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 699, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 940, in async_device_update
    await hass.async_add_executor_job(self.update)
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/xiaomi_cloud_map_extractor/camera.py", line 278, in update
    self._handle_map_data(map_name)
  File "/config/custom_components/xiaomi_cloud_map_extractor/camera.py", line 335, in _handle_map_data
    map_data, map_stored = self._device.get_map(map_name, self._colors, self._drawables, self._texts,
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/xiaomi_cloud_map_extractor/common/vacuum.py", line 27, in get_map
    response = self.get_raw_map_data(map_name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/xiaomi_cloud_map_extractor/common/vacuum.py", line 45, in get_raw_map_data
    map_url = self.get_map_url(map_name)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/xiaomi_cloud_map_extractor/common/vacuum_v2.py", line 18, in get_map_url
    if api_response is None or "result" not in api_response or "url" not in api_response["result"]:
                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable

Other info

No response

4ronos commented 11 months ago

I'm joining the add request!

4ronos commented 11 months ago

@PiotrMachowski, please tell me if there is any reason why support for this vacuum cleaner is not yet available? (s12/s10) Is it a matter of time, or do mijia servers not provide a card for it?

PiotrMachowski commented 11 months ago

@4ronos it is not a simple process "just download a map image from a URL". API has to be found and map has to be parsed from a binary file that has an unknown structure. And I have a lot of other repositories to maintain. If you can find an already existing implementation (even not in python) then it would be much much easier for me.

yopami commented 11 months ago

I need that too. Thanks.

nineteen0815 commented 10 months ago

I need that too. Thanks.

vadss commented 10 months ago

Have the same model. Anything i can do to help to speed up the fix?

daco77 commented 9 months ago

Hi, same vacuum for me.

vadss commented 9 months ago

It's the same model as Xiaomi Robot Vacuum S10 B106GL, i don't think they changed something.

casablancashub commented 8 months ago

Yeah, it would be great to have that integration available

Borty97 commented 7 months ago

I have the same vacuum and I would like to help! I'm very interested supporting this model

maksp86 commented 7 months ago

I`ve done some research and succesfully downloaded map from xiaomi.vacuum.c103
I guess that would work with some newest models as well The url for obtaining map file is https://api.io.mi.com/app/v2/home/get_interim_file_url_pro Object name is {user_id}/{device_id}/0 I only have one map on my vacuum, later I will try to create another one and check if it possible to download map by {user_id}/{device_id}/{map_name}

It seems like map file is not in zlib/gz format, maybe it is encrypted. At that point I cannot say it exactly

r-jean-pierre commented 7 months ago

I could hep too, I just received a xiaomi.vacuum.b106eu

Borty97 commented 7 months ago

@maksp86 wich is the format of the payload? The URL is correct? I get {"code":0,"message":"auth err"} JSON response (without autentication 😓). How do you authenticate? Shall we add the {user_id}/{device_id}/{map_name} endpoint?

maksp86 commented 7 months ago

Update: I've got map decrypted by reverse engineering the mihome plugin for xiaomi.vacuum.c103 (i guess that would work not only for my vacuum). I will post some code later. Now I have only one thing (some weird serial number) that I cant get from api

Some summary: encryption algorithm is a AES (ECB mode), pkcs7 padding Key is generated from serial_num+owner_id+device_id

After decryption I got hex string, that represents some zlib inflated file, so I think map is in viomi format

maksp86 commented 7 months ago

This is a reverse-engeneered map encryption algorithm

from Crypto.Cipher import AES
from Crypto.Hash import MD5
from Crypto.Util.Padding import pad, unpad
import base64

isEncryptKeyTypeHex = True

def aesEncrypted(data, key: str):
    cipher = AES.new(key.encode("utf-8"), AES.MODE_ECB)

    encryptedData = cipher.encrypt(
        pad(data.encode("utf-8"), AES.block_size, 'pkcs7'))

    encryptedBase64Str = base64.b64encode(encryptedData).decode("utf-8")
    return encryptedBase64Str

def aesDecrypted(data, key: str):
    parsedKey = key.encode("utf-8")
    if isEncryptKeyTypeHex:
        parsedKey = bytes.fromhex(key)

    cipher = AES.new(parsedKey, AES.MODE_ECB)

    decryptedBytes = cipher.decrypt(base64.b64decode(data))
    decryptedData = unpad(decryptedBytes, AES.block_size, 'pkcs7')
    decryptedStr = decryptedData.decode("utf-8")
    return decryptedStr

def md5key(string: str, model: str, device_mac: str):
    pjstr = "".join(device_mac.lower().split(":"))

    tempModel = model.split('.')[-1]

    if len(tempModel) == 2:
        tempModel = "00" + tempModel
    elif len(tempModel) == 3:
        tempModel = "0" + tempModel

    tempKey = pjstr + tempModel
    aeskey = aesEncrypted(string, tempKey)

    temp = MD5.new(aeskey.encode('utf-8')).hexdigest()
    if isEncryptKeyTypeHex:
        return temp
    else:
        return temp[8:-8].upper()

def genMD5key(wifi_info_sn: str, owner_id: str, device_id: str, model: str, device_mac: str):
    arr = [wifi_info_sn, owner_id, device_id]
    tempString = '+'.join(arr)
    return md5key(tempString, model, device_mac)

def unGzipCommon(data: bytes, wifi_info_sn: str, owner_id: str, device_id: str, model: str, device_mac: str):
    tempKey = genMD5key(wifi_info_sn, owner_id, device_id, model, device_mac)
    base64map = base64.b64encode(data)

    tempString = aesDecrypted(base64map, tempKey)
    return tempString

P.S functions naming was taken from original code Usage:

map_file = some_func_to_download_map() -> bytes
wifi_info_sn = "Weirdo looking string, that I got from attributes section of vacuum entity in Xiaomi Miot Auto"

unGzipCommon(map_file, wifi_info_sn, owner_id, device_id, model, device_mac)

maksp86 commented 7 months ago

@PiotrMachowski Think my research would help ;-)

maksp86 commented 7 months ago

vacuum_map_parser_dreame raised Unsupported frame type error vacuum_map_parser_viomi parsed without errors but it seems like nothing is parsed Also it prints in console "223371 bytes remained in the buffer"

_parsedmap output

vacuum_map_parser_roborock raised IndexError: index out of range error somewhere in parsing code

I can send you a decrypted map file if you want

xvolte commented 6 months ago

I m also very interested for this integration for the S12 Did you send the raw map? (It is listed in the first post as « unticked ») Thank you very much for your hard work! @PiotrMachowski @maksp86

xvolte commented 6 months ago

Could you please attach or publish somewhere then decrypted map file (before using any parser / parsing obviously) but already decrypted, to save time ? Thanks in advance

maksp86 commented 6 months ago

Could you please attach or publish somewhere then decrypted map file (before using any parser / parsing obviously) but already decrypted, to save time ? Thanks in advance

Here you go decrypted_encrypted_map_test.tar.gz

maksp86 commented 6 months ago

Also there is all files needed to download map from miot-based vacuum (3C Enchanced, S10/S12, ijai.vacuum.*) map-download-kit-for-miot-vacuums.tar.gz I`ve also added patch files for two components

xvolte commented 6 months ago

@

vacuum_map_parser_dreame raised Unsupported frame type error vacuum_map_parser_viomi parsed without errors but it seems like nothing is parsed Also it prints in console "223371 bytes remained in the buffer"

_parsedmap output

vacuum_map_parser_roborock raised IndexError: index out of range error somewhere in parsing code

I can send you a decrypted map file if you want

Maybe it is a dumb question but ... did you try the xiaomi map parser? what is the result ?

maksp86 commented 6 months ago

Maybe it is a dumb question but ... did you try the xiaomi map parser? what is the result ?

It fails on parsing.. at least for me on line 55 of xiaomi_cloud_map_extractor\xiaomi\map_data_parser.py
block_type = MapDataParserXiaomi.get_int16(header, 0x00)

xvolte commented 6 months ago

For information, the vacuum does not allow connection if the PC is on another subnet.... i connected in the same subnet, and now got a different error ...

now it fails here :

    if wifi_info_sn == None:
        raise Exception("Get wifi_info_sn failed")

it is strange as in your code, you seem to be working with a config option ? WIFI_INFO_SN

Traceback (most recent call last): File "C:\Users\XXXX\Downloads\xiaomimapextractor\master\map_downloader.py", line 59, in <module> main() File "C:\Users\XXXX\Downloads\xiaomimapextractor\master\map_downloader.py", line 35, in main raise Exception("Get wifi_info_sn failed") Exception: Get wifi_info_sn failed

maksp86 commented 6 months ago

Are you on latest version of python-miio? You should use lib from their master git branch

xvolte commented 6 months ago

yes i am. I have a xiaomi.vacuum.b106eu, maybe there are small adaptation to make, i don't know. but i don't have the wierd looking wifi string.... mine looks like this :

device.get_property_by(7, 45)[0]["value"]

'value': '[0,3,1,3,2,0,-3600,15,12.0,0,"fr_FR","xx23xxxxxxx46","en_US",0,-1,0,0,1,2,1]' --> i did replace the numbers with some xxx

The user_id is NOT present in this sentence, nor there is any semi column in the text. (i saw you are looking for semicolumn)

version of my robot is 4.3.3_0016

Tarh-76 commented 6 months ago

Here you go decrypted_encrypted_map_test.tar.gz

Thanks! The decrypted version definitely looks like zlib-compressed data to me. ... managed to decompress it 0_decrypted_uncompressed.tar.gz

Tarh-76 commented 6 months ago

Ok I managed to get this from your file: map

The map starts at offset 0x75 in plain 8bit "grayscale" format. I couldn't figure out where the dimensions are stored. And the dimensions are definitely not fixed as I extracted my own map which is 800x800

Borty97 commented 6 months ago

Is there any news on this issue?

maksp86 commented 6 months ago

Is there any news on this issue?

I'm currently spending free time trying to decompile mi home plugin code for data extraction algorithm, but it much harder for understanding to me than code for decryption

Tarh-76 commented 6 months ago

As for now, I'm trying to reverse the part after the map. I think it has something to do with the last cleaning route (repeated 15-byte structure starting with 120C08) after some kind of a header (271 bytes in my case and 27 bytes for maksp86's map - zones? ). Though maksp86 and I have different vacuum models (Mijia 3C and S10 respectively) the format of the map looks similar if not identical. UPD Now it came to me that starting at offset 0xB of this header the map name is being stored in UTF-8. "Хата" it is. And a byte at 0xA is the size of this string.

maksp86 commented 6 months ago

UPD Now it came to me that starting at offset 0xB of this header the map name is being stored in UTF-8. "Хата" it is. And a byte at 0xA is the size of this string.

About the end of the file between the cleaning routes and image there are room information with utf-8 encoded strings "Спальня", "Гостиная" and "Коридор" (Bedroom, Living room and Hallway respectively) Also idk if this is necessary, but image is flipped vertically, regarding what is shown in the application

xvolte commented 6 months ago

yes i am. I have a xiaomi.vacuum.b106eu, maybe there are small adaptation to make, i don't know. but i don't have the wierd looking wifi string.... mine looks like this :

device.get_property_by(7, 45)[0]["value"]

'value': '[0,3,1,3,2,0,-3600,15,12.0,0,"fr_FR","xx23xxxxxxx46","en_US",0,-1,0,0,1,2,1]' --> i did replace the numbers with some xxx

The user_id is NOT present in this sentence, nor there is any semi column in the text. (i saw you are looking for semicolumn)

version of my robot is 4.3.3_0016

Did i make any mistake? Anyone else having a S12 having the same issue ?

maksp86 commented 6 months ago

Did i make any mistake? Anyone else having a S12 having the same issue ?

For now just hardcode your string into wifi_info_sn But I think this string is on the same index in array so we can get it by constant index and not try to search it

Tarh-76 commented 6 months ago

About the end of the file between the cleaning routes and image there are room information with utf-8 encoded strings "Спальня", "Гостиная" and "Коридор" (Bedroom, Living room and Hallway respectively) Also idk if this is necessary, but image is flipped vertically, regarding what is shown in the application M-hm. Thanks. Also, before a zone name the string length is located. And before that - zone id on the map ("color index") and a byte 0x12 (wtf is 0x12?). To sum it up, a zone descriptor now looks like the following structure: { unknown_uint32 : uint32 //some value changing from descriptor to descryptor, zero for the first zone in list (offset?) unknown_62 : byte //always 62 unknown: byte // in my file either 28 or 22 unknown_08: byte //always 08 zone_id: byte unknown_12: byte //always 12 zone_name_size: byte zone_name: byte[] //utf8 encoded string unknown_0018: uint16 //always 18 00 unknown_0020: uint16 //always 20 00 unknown_0028: uint16 //always 28 00 unknown_42: byte //always 42 unknown_0A: byte // always 0A unknown_0D: byte // always 0D unknown_bytes: byte[7] // some bytes changing from descriptor to descriptor } I think if we want further progress we should debug this dynamically. (e.g., change cleaning mode -> look at the changes in the data downloaded)

Tarh-76 commented 6 months ago

device.get_property_by(7, 45)[0]["value"] 'value': '[0,3,1,3,2,0,-3600,15,12.0,0,"fr_FR","xx23xxxxxxx46","en_US",0,-1,0,0,1,2,1]' --> i did replace the numbers with some xxx The user_id is NOT present in this sentence, nor there is any semi column in the text. (i saw you are looking for semicolumn) version of my robot is 4.3.3_0016

Did i make any mistake? Anyone else having a S12 having the same issue ?

Nah, no mistake, almost the same on S10. Might be if you take a closer look at "xx23xxxxxxx46" the semicolumn "hides" in the middle? That was exactly my case.

Tarh-76 commented 6 months ago

BTW are you still able to acquire and decrypt the map? The thing is I tried this recently and eventually failed.

xvolte commented 6 months ago

Nah, no mistake, almost the same on S10. Might be if you take a closer look at "xx23xxxxxxx46" the semicolumn "hides" in the middle? That was exactly my case.

Verified again tonight : NO semicolumn.... can you tell me the length of the wifiinfosn ? maybe if i take only the correct amount of chars it will be ok ....

mine is 18 characters long.

if i hardcode this directly, the script fails at this step :

map_downloader.py", line 49, in main deflated_map = bytes.fromhex(hexstr) ^^^^^^^^^^^^^^^^^^^^^ TypeError: fromhex() argument must be str, not bytes

So it seems to be getting the encrypted map, however, it can't deflate it

maksp86 commented 6 months ago

@xvolte

it is strange as in your code, you seem to be working with a config option ? WIFI_INFO_SN

I`m sorry, didn't notice this before posting

Verified again tonight : NO semicolumn....

I think presence of semicolumn in this field differs between vacuums/account/regions In mine case my vacuum is Chinese region only, Mi Home region is Mainland China, but Mi Account is in Russian region

mine is 18 characters long.

Yea, mine is 18 chars too... Is it in uppercase in your case?

map_downloader.py", line 49, in main deflated_map = bytes.fromhex(hexstr) ^^^^^^^^^^^^^^^^^^^^^ TypeError: fromhex() argument must be str, not bytes

It means that decryption has failed, because decryption algorhytm is supposed to return a hex string, but here it is failing to transform hex to bytes

@Tarh-76

I think if we want further progress we should debug this dynamically. (e.g., change cleaning mode -> look at the changes in the data downloaded)

I can send some map files during different stages of cleaning if you wish

The thing is I tried this recently and eventually failed.

On what stage the script fails?

maksp86 commented 6 months ago

Here is updated script map_downloader.py.gz

Changelog:

added get_map_url into file to prevent unnecessary import of ViomiVacuum, so vacuum_v2.py.patch is no longer required
removed using WIFI_INFO_SN constant what I left previously by mistake
saving now into "maps" folder
also saving unpacked map
script now will try to find wifi_info_sn in properties

Tarh-76 commented 6 months ago

The thing is I tried this recently and eventually failed.

On what stage the script fails?

Decryption. The data is being downloaded fine, key building is ok. But the result of aes decryption is not as expected. It looks like wrong key/key generation algorithm have changed or something but how can it be? It worked a week ago and now it does not. Actually I wrote my own script based on yours and integrated it into my HA. I'll try your version "as is" later today.

UPD I tested your script (with a minor bugfix - removed excessive base64 encoding and made wifi_info_sn lowercase ) and it has shown the same result - decryption failed. But after I had started and stopped cleaning (via phone), the result changed and both our scripts started decrypting the data correctly.

Tarh-76 commented 6 months ago

I can send some map files during different stages of cleaning if you wish

That would be nice, indeed

maksp86 commented 6 months ago

I can send some map files during different stages of cleaning if you wish

That would be nice, indeed

There is some decrypted&decompressed map files on different stages of cleaning with screenshots of Mi Home for better understanding. I hope it helps maps_on_different_stages.tar.gz

xvolte commented 5 months ago

Here is updated script map_downloader.py.gz

Changelog:

* added get_map_url into file to prevent unnecessary import of ViomiVacuum, so vacuum_v2.py.patch is no longer required

* removed using WIFI_INFO_SN constant what I left previously by mistake

* saving now into "maps" folder

* also saving unpacked map

* script now will try to find wifi_info_sn in properties

Thank you. I also noticed since my model is b106eu, that the file map_decrypter.py which contain AES temp key would be invalid if i append the mac address of the device + the device name (which is b106eu), as the AES key would be 18 and not 16...

i did try to crop the model name to b106, but it doesn't seems to work .... here is the code that needs to be modified (with the [:4]): tempKey = pjstr + tempModel[:4] but then the decryption would fail ... so i'm unsure how to generate the correct key ...

xvolte commented 5 months ago

https://github.com/PiotrMachowski/Home-Assistant-custom-components-Xiaomi-Cloud-Map-Extractor/issues/491

Looks like we are working on the issue in this thread for both S12 (B106EU, B106GL, B106CN) and also for C103

xvolte commented 5 months ago

Is there any news on this issue?

I'm currently spending free time trying to decompile mi home plugin code for data extraction algorithm, but it much harder for understanding to me than code for decryption

Can you tell me how you did it, so I can have a look for the S12 ? I think the key generation is not the same on S12 (model B106EU).

xvolte commented 5 months ago

Is there any news on this issue?

I'm currently spending free time trying to decompile mi home plugin code for data extraction algorithm, but it much harder for understanding to me than code for decryption

Can you tell me how you did it, so I can have a look for the S12 ? I think the key generation is not the same on S12 (model B106EU).

Hello @maksp86 and @Tarh-76 :) I need your help to progress and identify how the key is generated for b106eu. can you tell me how you decompiled mi home plugin? I know we are all spending free time and you already made very good progress so i thank you a lot for you work! Thanks in advance

maksp86 commented 5 months ago

Hello @maksp86 and @Tarh-76 :) I need your help to progress and identify how the key is generated for b106eu. can you tell me how you decompiled mi home plugin? I know we are all spending free time and you already made very good progress so i thank you a lot for you work! Thanks in advance

You can send me plugin using vevs modded mi home To do so:

Open Vevs Mi Home
Long tap on your vacuum -> more -> rename -> remember Plugin ID (if there no that one, click on vacuum and wait to load)
Go to profile -> additional settings -> laboratory
Write custom log files -> grant access
Delete plugins (long press)
Restart mi home
Open your vacuum page
Go to /sdcard/vevs/files/plugin/install/rn/(Plugin ID from the first paragraph)/android/
Send here your main.bundle

xvolte commented 5 months ago

Thank you very much, I just need to retrieve an android device ;)

Borty97 commented 5 months ago

I have Mi Home app (not vevs mmodded mi home? What is this???)

Anyway, I can not find vacuum -> more -> rename -> remember Plugin ID (when renaming only a dialog appears to change the name)

Go to profile -> additional settings -> laboratory (In additional settings I only have experimental features but only to link with Alexa)

I have a Samsung Galaxy S3 tab and iOS device. Neither of them have this options

PiotrMachowski / Home-Assistant-custom-components-Xiaomi-Cloud-Map-Extractor