Add support for Smart Scale

[jlboygenius]

I really appreciate the work done here. I've been using this for a long time to pull data into influxdb/grafana. Recently, I've decided to move over to using Home Assistant.

I keep getting this log warning: Unknown device ESF00+ Logger: pyvesync.vesync Source: /usr/local/lib/python3.8/site-packages/pyvesync/vesync.py:174

Any way to add in this device (it's a scale)?

I have packet captures of the app calling the "/cloud/v1/deviceManaged/fatScale/getWeighData" if needed.

[webdjoe]

If you send the complete packet capture data, I will work on adding this

[jlboygenius]

POST /cloud/v1/deviceManaged/fatScale/getWeighData HTTP/1.1 accept: application/json tk: XrXvMIej5tLnkZER9**Am4g== accountid: 1***3 tz: America/New_York accept-language: en appversion: 3.0.20 Content-Type: application/json Content-Length: 425 Host: smartapi.vesync.com Connection: Keep-Alive Accept-Encoding: gzip User-Agent: okhttp/3.12.1

{"traceId":"1600269682529","method":"getWeighData","token":"XrXv**N6d1QllmAm4g==","accountID":"13","timeZone":"America/New_York","acceptLanguage":"en","appVersion":"3.0.20","phoneBrand":"SM-G973U1","phoneOS":"Android 10","startTime":0,"endTime":1600269683,"configModule":"WiFiBT_Scale_FatScalePlus_US","cid":"0L**WOQMU8kZT-0fGZMCwA1","pageSize":100,"order":"desc","index":0,"flag":1}HTTP/1.1 200 OK Date: Wed, 16 Sep 2020 15:21:23 GMT Content-Type: application/json;charset=UTF-8 Transfer-Encoding: chunked Connection: keep-alive Server: Apache-Coyote/1.1

384 {"traceId":"1600269682529","code":0,"msg":"请求成功","result":{"data":[{"weigh_kg":83.5,"weigh_lb":184.1,"impedence":572.0,"timestamp":1600257968,"unit":"lb","timezone15m":"-20","arithmeticVersion":2,"gender":"2","age":39,"heightCm":178.0,"ID":6858643,"userId":"001"},{"weigh_kg":83.85,"weigh_lb":184.9,"impedence":552.0,"timestamp":1600006684,"unit":"lb","timezone15m":"-20","arithmeticVersion":2,"gender":"2","age":39,"heightCm":178.0,"ID":6779757,"userId":"001"},{"weigh_kg":84.8,"weigh_lb":187.0,"impedence":557.0,"timestamp":1599763345,"unit":"lb","timezone15m":"-20","arithmeticVersion":2,"gender":"2","age":39,"heightCm":178.0,"ID":6707719,"userId":"001"},{"weigh_kg":84.7,"weigh_lb":186.7,"impedence":553.0,"timestamp":1599666717,"unit":"lb","timezone15m":"-20","arithmeticVersion":2,"gender":"2","age":39,"heightCm":178.0,"ID":6683037,"userId":"001"}],"pageSize":100,"index":1599666717}} 0

[mstanislav]

Funny enough, I just started working on this a few minutes ago for my scale (ESF14) and have a (very rough) PoC going now, @jlboygenius.

➜  pyvesync git:(master) ✗ python3 test.py
Device Name:... Scale
Model: ........ ESF14
Subdevice No: . None
Status: ....... off
Online: ....... offline
Type: ......... BT-Scale
UUID: ......... ED:67:38:RE:DA:CT
Weight : ...... 88750.0 grams
Date : ........ 2020-12-02 13:45:52

Was going to work a bit more on it and then throw a PR in as there's likely some TLC it will need, but the codebase was rather friendly so I tried to keep most things normalized to existing integrations best I could.

[mstanislav]

Here's a branch on my fork that has the above functionality working for my specific scale model. I decided not to do a PR because there's some fragility here for sure. https://github.com/mstanislav/pyvesync/tree/esf14

The scale weight comes from a pagination in an array of results. I don't have a wild amount of weigh-ins, but the default API call has a page = 1 and pageSize = 100. For @jlboygenius's output above, mine did not have an order parameter. I just tried this and it didn't change the ordering with different values (e.g. no value, asc, desc). Because of this, I am currently just taking the last array element to grab the most recent data... this is likely to break after enough results. Unclear how large you can make pageSize off hand.

Of note, my device does not seem to provide a CID, so I had to patch core files related to that. I also had an issue with a key trying to display that was type None, so added a patch for that as well.

Happy to provide more data/time on this if it's worthwhile @webdjoe.

[webdjoe]

Thank you both for contributing. @jlboygenius can you please send packets from the device list screen, just as a double check?

@mstanislav can you send me all of your captures for the scale, including the device list, scale screen and another other associated screens? I'm really curious to see how the calls are handled without a CID. The briefly looked at your code and it's a great start so just submit a PR and we will take it from there

You can email me directly if it is easier at webdjoe gmail.com

One of my To-Do's is to figure out a way to handle adding devices more gracefully in the core. It's too haphazard for my liking. It looks like moving away from using the CID is going to be on the list as well.

I also noticed there are 9 different etekcity scales on their website. What model are each of you using?

It looks like @jlboygenius your packet captures are streaming data from the vesync server, which I assume means you are standing on the scale in the app? Are there any packet captures that contain the history? It would be easier to just download the history than to continuously poll an endpoint and filter out the 0 weights, especially in HA.

[mstanislav]

These seem to be the important API calls related to the device usage/statistics/configuration. I've redacted my accountID, token, and macID/UUID, but the rest is unchanged. The first two use v2 API endpoints, but the last uses the v1 API still as one minor note. I think that's all the useful data I see myself interacting with the app, but ping me if you want to know anything else @webdjoe!

ESF14 is the model https://static.etekcity.com/files/manual/ESF14_WEB_manual_US_en.pdf

POST: https://smartapi.vesync.com/cloud/v2/deviceManaged/devices

HEADERS
User-Agent:       VeSync/VeSync 3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=UTF-8
Content-Length:   307
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "appVersion": "VeSync 3.0.40",
    "debugMode": false,
    "method": "devices",
    "pageNum": 1,
    "pageSize": 100,
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "timeZone": "America/New_York",
    "token": "REDACTED",
    "traceId": "1607130384181"
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": {
        "list": [
            {
                "authKey": null,
                "cid": null,
                "configModule": "BT_Scale_ESF14_US",
                "connectionStatus": "offline",
                "connectionType": "bleBeacon",
                "currentFirmVersion": "0",
                "deviceFirstSetupTime": "Nov 16, 2020 9:57:24 PM",
                "deviceImg": "https://image.vesync.com/defaultImages/ESF14_Series/ESF14_240.png",
                "deviceName": "Scale",
                "deviceRegion": "US",
                "deviceStatus": "off",
                "deviceType": "ESF14",
                "extension": null,
                "isOwner": true,
                "macID": "REDACTED",
                "mode": null,
                "speed": null,
                "subDeviceNo": null,
                "type": "BT-Scale",
                "uuid": "REDACTED"
            }
        ],
        "pageNo": null,
        "pageSize": 100,
        "total": 1
    },
    "traceId": "1607130384181"
}

POST: https://smartapi.vesync.com/cloud/v2/deviceManaged/getWeighingDataV2

HEADERS
User-Agent:       VeSync/VeSync 3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=UTF-8
Content-Length:   385
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "allData": true,
    "appVersion": "VeSync 3.0.40",
    "configModule": "BT_Scale_ESF14_US",
    "debugMode": false,
    "deviceRegion": "US",
    "method": "getWeighingDataV2",
    "page": 1,
    "pageSize": 100,
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "timeZone": "America/Detroit",
    "token": "REDACTED",
    "traceId": "1607130394820"
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": {
        "weightDatas": [
             {
                "age": 35,
                "appleHealthStatus": 0,
                "arithmeticVersion": 1,
                "bfr": null,
                "configModule": "BT_Scale_ESF14_US",
                "fitbitStatus": 0,
                "gender": "2",
                "googleFitStatus": 0,
                "heightCm": 183.0,
                "impedance": 52394.0,
                "isManualInput": false,
                "macID": "REDACTED",
                "subUserID": null,
                "timestamp": 1607003967,
                "uploadTimestamp": "2020-12-03 13:59:29",
                "weightG": 87750.0
            },
            {
                "age": 35,
                "appleHealthStatus": 0,
                "arithmeticVersion": 1,
                "bfr": null,
                "configModule": "BT_Scale_ESF14_US",
                "fitbitStatus": 0,
                "gender": "2",
                "googleFitStatus": 0,
                "heightCm": 183.0,
                "impedance": 57956.0,
                "isManualInput": false,
                "macID": "REDACTED",
                "subUserID": null,
                "timestamp": 1607089558,
                "uploadTimestamp": "2020-12-04 13:45:59",
                "weightG": 87650.0
            }
        ]
    },
    "traceId": "1607130394820"
}

POST: https://smartapi.vesync.com/cloud/v1/deviceManaged/bleDeviceConfigurations

HEADERS
tk:               REDACTED
accountID:        REDACTED
tz:               America/Detroit
Accept-Language:  en
User-Agent:       VeSync/3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=utf-8
Content-Length:   397
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "appVersion": "VeSync 3.0.40",
    "configModule": "BT_Scale_ESF14_US",
    "debugMode": false,
    "macID": "REDACTED",
    "method": "bleDeviceConfigurations",
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "timeZone": "America/Detroit",
    "token": "REDACTED",
    "traceId": "1607131609514",
    "uuid": "REDACTED"
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": {
        "address": "http://fw.vesync.com/BT_Scale_ESF14_US/1.0.01",
        "defaultDeviceImg": "https://image.vesync.com/defaultImages/ESF14_Series/ESF14_80.png",
        "lastFirmVersion": "1.0.01"
    },
    "traceId": "1607131609514"
}

[jlboygenius]

Here's the device info for the scale. from /cloud/v2/deviceManaged/devices

{ "deviceRegion": "US", "isOwner": true, "deviceName": "Fitness Scale", "deviceImg": "https://image.vesync.com/defaultImages/ESF00__Series/icon_fitness_scale_160.png", "cid": "1", "connectionType": "wifi+BT", "deviceType": "ESF00+", "type": "BT-Scale", "uuid": "24::E6", "configModule": "WiFiBT_Scale_FatScalePlus_US", "macID": "24:****:E6", "currentFirmVersion": "1.0.01", "deviceFirstSetupTime": "Jun 19, 2019 10:37:35 PM" },

[webdjoe]

Thank you both for that information. I believe @mstanislav your device does not have a cid because it is not wifi enabled, it only communicates with the app, which then uploads it.

There is a button in pictures of the app that says 'view data graph', does this show historical data?

These api's are much more different from each other than I expected. Each one would require it's own object. @jlboygenius if you upgrade the app, is there any change?

@mstanislav I see subUserId: null on your captures. Is it possible to add additional users to be weighed?

@jlboygenius There is a startDate and endDate on your request for the weighing data. Do you enter this information somewhere or does it take the last weigh data into account? I ask because the end date was back in September, was this when you captured these packets?

[jlboygenius]

Yes, the start dates are from when I logged the data, back in September.

I'm using the packet capture app on my phone. I don't think vesync likes it, because the app doesn't work right when it's capturing packets. I'm unable to capture packets showing the historical data.

[mstanislav]

One thing I noticed about the getWeighingDataV2 is that there is a call made by the same view, but it passes in a uploadTimestamp key in the JSON (e.g. value of 2020-12-07 21:07:54) which I am guessing is pulling the last record timestamp from the local database so the backend can try to see if it needs any new data locally. In my case, results are empty on the built-in request (as expected) since my device has it all.

I edited a request to change this date and it returned records after my earlier date as expected. If there's a way to cache that value (last timestamp) you could then call a request with that to find newer entries!

---

There is a button in pictures of the app that says 'view data graph', does this show historical data?

When you load the graph display is where the https://smartapi.vesync.com/cloud/v2/deviceManaged/getWeighingDataV2API is called, which in my experience returns all the data based on those pagination parameters. I'm guessing it's managing a SQL database locally and when it sees new data as I can view it all in airplane mode just fine.

I see subUserId: null on your captures. Is it possible to add additional users to be weighed?

It turns out there is! I added a second user and did a weigh-in, here are those request captures. Interestingly enough, there's a CID (which is the MAC address in my case) populated with the second user's data upload, but I don't see it used in other queries.

---

Here are the queries related to creating a new user, adding a weight reading, retrieving data, and switching users.

POST: https://smartapi.vesync.com/cloud/v2/user/createSubUserV2

HEADERS
User-Agent:       VeSync/VeSync 3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=UTF-8
Content-Length:   446
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "appVersion": "VeSync 3.0.40",
    "birthday": "1984/06/15",
    "configModule": "BT_Scale_ESF14_US",
    "debugMode": false,
    "deviceRegion": "US",
    "gender": "1",
    "heightCm": 170,
    "method": "createSubUserV2",
    "nickname": "Test",
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "targetBfr": 0,
    "timeZone": "America/Detroit",
    "token": "REDACTED",
    "traceId": "1607375051366",
    "weightTargetG": 0
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": {
        "subUserID": "000001"
    },
    "traceId": "1607375051366"
}

POST: https://smartapi.vesync.com/cloud/v1/log/uploadAppLog

HEADERS
User-Agent:       VeSync/VeSync 3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=UTF-8
Content-Length:   928
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "appTimestamp": "1607375274",
    "appVersion": "VeSync 3.0.40",
    "cid": "REDACTED",
    "configModule": "BT_Scale_ESF14_US",
    "debugMode": false,
    "deviceRegion": "US",
    "logs": [
        {
            "age": 36,
            "arithmeticVersion": 1,
            "bfr": 0.0,
            "bmi": 10.5,
            "bmr": 0,
            "bodyWater": 0.0,
            "boneMass": 0.0,
            "configModule": "BT_Scale_ESF14_US",
            "fitbitStatus": 0,
            "gender": "1",
            "googleFitStatus": 0,
            "heightCm": 170.0,
            "id": 0,
            "impedance": 0,
            "leanFatWeight": 0.0,
            "macID": "REDACTED",
            "muscleMass": 0.0,
            "physiologicalAge": 0,
            "protein": 0.0,
            "samsungHealthStatus": 0,
            "skeletalMuscle": 0.0,
            "status": 1,
            "subUserID": "000001",
            "subcutaneousFat": 0.0,
            "timestamp": 1607375267,
            "visceralFat": 0.0,
            "weightG": 30300.0
        }
    ],
    "macID": "REDACTED",
    "method": "uploadAppLog",
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "timeZone": "America/Detroit",
    "token": "REDACTED",
    "traceId": "1607375274171",
    "type": "deviceEvents"
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": null,
    "traceId": "1607375274171"
}

POST: https://smartapi.vesync.com/cloud/v2/deviceManaged/uploadWeighingDataV2

HEADERS
User-Agent:       VeSync/VeSync 3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=UTF-8
Content-Length:   569
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "appVersion": "VeSync 3.0.40",
    "configModule": "BT_Scale_ESF14_US",
    "debugMode": false,
    "deviceRegion": "US",
    "method": "uploadWeighingDataV2",
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "timeZone": "America/Detroit",
    "token": "REDACTED",
    "traceId": "1607375274181",
    "weightDatas": [
        {
            "age": 36,
            "arithmeticVersion": 1,
            "configModule": "BT_Scale_ESF14_US",
            "gender": "1",
            "heightCm": 170.0,
            "impedance": 0.0,
            "macID": "REDACTED",
            "subUserID": "000001",
            "timestamp": 1607375267,
            "weightG": 30300.0
        }
    ]
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": {
        "uploadTimestamp": "2020-12-07 21:07:54"
    },
    "traceId": "1607375274181"
}

POST: https://smartapi.vesync.com/cloud/v2/user/getOneSubUserV2

HEADERS
User-Agent:       VeSync/VeSync 3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=UTF-8
Content-Length:   365
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "appVersion": "VeSync 3.0.40",
    "configModule": "BT_Scale_ESF14_US",
    "debugMode": false,
    "deviceRegion": "US",
    "method": "getOneSubUserV2",
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "subUserID": "000001",
    "timeZone": "America/Detroit",
    "token": "REDACTED",
    "traceId": "1607375630142"
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": {
        "subUsers": {
            "birthday": "1984/06/15",
            "gender": "1",
            "heightCm": 170.0,
            "nickname": "Test",
            "photo": null,
            "subUserID": "000001",
            "targetBfr": 0.0,
            "weightTargetG": 0.0
        }
    },
    "traceId": "1607375630142"
}

POST: https://smartapi.vesync.com/cloud/v2/user/getAllSubUserV2

HEADERS
User-Agent:       VeSync/VeSync 3.0.40(LG-VS450PP;Android 4.4.2)
Content-Type:     application/json; charset=UTF-8
Content-Length:   373
Host:             smartapi.vesync.com
Connection:       Keep-Alive
Accept-Encoding:  gzip

REQUEST
{
    "acceptLanguage": "en",
    "accountID": "REDACTED",
    "appVersion": "VeSync 3.0.40",
    "configModule": "BT_Scale_ESF14_US",
    "debugMode": false,
    "deviceRegion": "US",
    "isAsc": false,
    "method": "getAllSubUserV2",
    "pageSize": 300,
    "phoneBrand": "LG-VS450PP",
    "phoneOS": "Android 4.4.2",
    "timeZone": "America/Detroit",
    "token": "REDACTED",
    "traceId": "1607375748523"
}

RESPONSE
{
    "code": 0,
    "msg": "请求成功",
    "result": {
        "subUsers": [
            {
                "birthday": "1984/06/15",
                "gender": "1",
                "heightCm": 170.0,
                "nickname": "Test",
                "photo": null,
                "subUserID": "000001",
                "targetBfr": 0.0,
                "weightTargetG": 0.0
            }
        ]
    },
    "traceId": "1607375748523"
}

[webdjoe]

That helps a lot. I will post when I have made some more progress

[webdjoe]

@mstanislav I have a rough draft of the class to control your scale. @jlboygenius I haven't forgotten about you, I'll have something in a couple of days.

@mstanislav check out the fork here https://github.com/webdjoe-bot/pyvesync

Two options to return data - json data for subuser in kg or lbs by vsscale.user_json_kg(user=0) and vsscale.user_json_lb(user=0) Make sure to run update() and vsscale.get_details() just to be sure. If there are no subusers it will default to the only user available. Let me know how this works

[nathan7432]

Hey @webdjoe just wanted to add a +1 for smart scale support! I have a Etekcity ESF24. I'll be taking a look at your code the next couple of days to see how you communicate with the API and such.

[alfista2600]

interested as well!

[lhassell]

Any luck with this? I have a different model, but same brand.