battotalenergycharged & battotalenergydischarged only showing full kWh

[RCGrahamWilmott]

Both battotalenergycharged and battotalenergydischarged are only returning full kWh on the energy dashboard, rather than showing in W or fractions of kWh. This leads to strange spikes in the graph:

The https://m.ginlong.com/ website shows battery charge/discharge in W. I think it might be a datatype issue (integer rather than float?) but not 100% sure.

[hultenvp]

Hi @RCGrahamWilmott ,

I don't have a unit with battery support myself, so I rely on the input provided in issue #16 . For now I provide only batcapacityremaining (key: 1cv, %) , battotalenergycharged(key: 1cx, kWh) and battotalenergycharged(key: 1cy, kWh). I do read them as floats with 2 digits precision, so I'd expect them to show up as sensor providing values with 2 digits precision. I suspect that the energy dashboard calculates the difference with the previous value every time it receives a new one causing the spikes. Could that be the case? You can see that in the history as infrequent increments.

The battery charge/discharge you see on ginlong are probably a combination of keys 1ff and 1ct. That would give power figures with unit Watt. All others are energy in kWh.

[RCGrahamWilmott]

Hi @hultenvp

I'm happy to help you try and debug this. I assume that you are getting JSON out of https://m.ginlong.com/? If you talk me through the steps, I will happily send you anything I can.

Looking at my https://m.ginlong.com/ page, I can see that "Total Energy Charged" and "Total Energy Discharged" are being rounded down to whole kWh. Interestingly, "Daily Energy Charged", "Monthly Energy Charged" and "Yearly Energy Charged" are showing to the nearest 100w.

My history in home assistant also shows updates only happening on whole kWh.

Looking at https://m.ginlong.com/main.html?class=plant&subClass=overview&page=view

Looking at https://m.ginlong.com/main.html?class=device&subClass=inverter&page=view

Looks like it might be possible to get the grid consumption / return to grid as well which can also go into the home assistant energy page for those without CT clamps / other ways of getting the data.

[hultenvp]

Hi @RCGrahamWilmott ,

The output is indeed json. I once found a post from someone who figured out the backend API, based my integration on that. Recently found out about this project: https://github.com/dkruyt/ginlong-scraper. Did not het around using it yet, but the original requester of the battery data used that scraper to provide me the input. I suggest you use that if you want more detailed info about the json payload ginlong returns.

Must admit I never really looked at the precision of the returned values. Just made it a 2 decimal float assuming it was identical to all other values the handler processes. The good thing is the energy dashboard does not care what energy data you feed it, it does not require the data to be ever-increasing; it can also deal with, for example, nightly resets. So we can try using the daily energy figures.

Could you try the following hack: Go to platform2_portal.py and make the following change in the dict (table) PORTAL_INVERTER_CONST: For the last 2 entries change "1cx" into "1cz" and change "1cy" into "1da"? This will change the respective sensors to start using the daily charging/discharging figures which should have an accuracy of 100 Wh.

If that works better for you then I'll extend the integration to also support daily energy.

Also happy to add more entries if they help you calculating grid consumption/ return to grid, just let me know which ones you need.

Cheers

[RCGrahamWilmott]

Hi @hultenvp

I've changed platform2_portal.py to the following and rebooted:

'BAT_TOTAL_ENERGY_CHARGED': ['kWh', '1cz', float, 2], 'BAT_TOTAL_ENERGY_DISCHARGED': ['kWh', '1da', float, 2],

I will keep an eye on it today (a very cloudy day, so not sure I'll get much into the battery) and let you know what I see.

For grid energy - I think "Daily Grid Feed-in" and "Daily Energy Purchased" would all be useful and allow people to fully use the energy tab.

(and maybe "Daily Consumption", which although not needed for the energy tab might be useful / interesting to some.)

[hultenvp]

Looking forward to your results.

For the grid energy: could you help by providing the Json? I do not see it in the payload, likely because I don't have the battery.

Cheers

[RCGrahamWilmott]

Hi

Using "1cz" and "1da" is now giving me my battery charge/discharge in 100w increments and the graph on the energy tab looks much better. Not sure if you want to hard-code this change or add two new sensors?

For JSON - I'm having issues getting anything out of m.ginlong.com - either a blank (0byte) response or an error 400/403 and I can't quite work out what I'm doing wrong. Any advice?

[hultenvp]

I will add 2 new sensors to the integration. Until that time you can continue using the hack. Are you using the ginlong scraper for the JSON? If not, then I highly recommend you do use that one. 403 looks like a request without being logged in, so please double check your username and password. If that does not work we can add some lines to the integration, but that produces a lot of logging output for HA to deal with, so better not if we can prevent it. ;-)

[hultenvp]

I made a new branch with the daily stats included. It also shows you the JSON if you put log level to debug in your configuration.yaml: logger: default: warning logs: custom_components.solis: debug

It will provide a raw, unformatted dump of the JSON, but that can be fixed with JSON pretty print: https://jsonformatter.org/json-pretty-print Best to disable the logging immediately again, it will fill up your disk with log data faster than you think if you leave it in. Don't forget to delete your personal data.

Code can be found here: https://github.com/hultenvp/solis-sensor/archive/refs/heads/23-add-more-sensors-for-battery.zip

Once we know what keys to add I'll add them here and merge the branch.

Thanks for your help so far.

[RCGrahamWilmott]

Thanks (again) for everything.

I've got the following JSON for you. I changed "za", "zc" & "g" as they seem to contain personal data. I think everything else is OK... ("zh" looks like another IP, but not one of mine. I guess this is where it is sending data?)

{ "_st": "1", "1af": "242.30", "1ag": "0.00", "1ah": "0.00", "1ai": "1.00", "1aj": "0.00", "1ak": "0.00", "1ao": "0", "1ar": "50.00", "za": "MY-LOGGER-NUMBER", "1aw": "1.80", "ah": "4960", "zc": "MY.IP.ADDRESS.AND:PORT", "zd": 1635438189000, "2tm": "100", "ze": 1635438190000, "zf": "104312770", "zg": "C1,16", "2tp": "-7", "zh": "47.88.8.200:10000", "zi": "02", "ap": "0000", "aq": "0000", "zk": "1", "ar": "0000", "2tt": "48.96", "zl": "0", "as": "0000", "1bc": "37.00", "2tu": "6.60", "at": "0000", "1bd": "2.60", "1be": "37.00000000000001", "2tv": "37.00", "1ru": "0", "1bf": "37.00000000000001", "2tw": "37.00", "1rw": "0", "1bj": "242.20", "1bm": "2.96", "_ut": "1635438200", "1bq": "0", "1br": "0", "1bs": "240", "1bt": "1.00", "1bu": "4", "1bv": "113", "1bw": "0.00", "1bx": "1.20", "1by": "121.7", "1bz": "206.70000000000005", "1sm": "0036", "_et": "1635441790", "1ca": "121.7", "1ki": "0026", "1cb": "206.70000000000005", "1a": "159.50", "1b": "166.40", "1cj": "310.0", "1j": "0.00", "1k": "0.00", "1cn": "52", "1co": "3.10", "1cp": "52.400000000000006", "2vb": "6.70", "1cq": "52.400000000000006", "_": "0000", "2vc": "1.10", "2vd": "2.10", "1cr": "49.00", "2ve": "242.30", "a": "01", "1cs": "6.40", "stat": "0", "b": "0505", "1ct": "310", "2vg": "10", "c": "719987", "d": "1258", "1cv": "46", "e": "1634718202", "f": "0001", "1cx": "19", "g": "INVERTOR", "1cy": "22", "1cz": "1.90", "1lf": "G98", "1da": "1.10", "1db": "19.000000000000004", "1dc": "22.600000000000005", "ruleCode": "0102010505", "1dd": "19.000000000000004", "1ll": "0001", "1de": "22.600000000000005", "1df": "25.70", "94ax": "100", "4ke": "1", "dt": "1635438189000", "1fe": "2", "1ff": "2", "1vw": "21", "1vx": "10", "1vy": "28", "1vz": "17", "1wa": "22", "1wb": "36", "1fs": "", "7cf": "0.00", "7cg": "0.00", "1gr": "00F5", "7ch": "0.00", "7ci": "0", "13ah": "25.70", "a0": "1", "1hk": "3600" }

I can turn logging on again if you need more. And/or try and decode some of these values by looking at the website and trying to match values.

[hultenvp]

Thanks Graham,

Part of the JSON payload seems missing. I actually need the part where keys are related to English names. The exact logic of the JSON eludes me, but you probably sent only the dataJSON section. There are other sections and the grid info can be found in one of them. What I did is I copied the whole JSON to https://jsonformatter.org/json-pretty-print. I clicked the wrench icon to repair the JSON (it expects double-quoted strings) and then clicked repair. Then you see in the right pane nicely formatted JSON. There you can find sections like the following:

      "realTimeDataElect": [
        {
          "baseUnit": "W",
          "key": "1bq",
          "name": "Power Grid Total Power",
          "unit": "W",
          "value": "67"
        },
        {
          "baseUnit": "Var",
          "key": "1br",
          "name": "Power Grid Total Reactive Power",
          "unit": "Var",
          "value": "67"
        },
        {
          "baseUnit": "VA",
          "key": "1bs",
          "name": "Power Grid Total Apparent Power",
          "unit": "VA",
          "value": "67"
        },
        {
          "key": "1bt",
          "name": "Grid Power Factor",
          "value": "0.00"
        }
      ],

There should be a section describing the keys grid info you need. For example from the screenshot above there should be a section describing what key belongs to "Power Grid Status".

You can always copy the whole JSON in, but you'll notice your private info is everywhere.

Cheers, Peter

[RCGrahamWilmott]

I turned logging back on and waited a while, and was only getting the same data in the logs.

I changed some code in platform2_portal.py to: _LOGGER.debug("Data received: %s", data) _LOGGER.debug("Data received RESULT: %s", portaldata['result']) _LOGGER.debug("Data received DEVICEWAPPER: %s", portaldata['deviceWapper']) _LOGGER.debug("Data received DATAJSON: %s", portaldata['dataJSON'])

This gives more data in the logs, as well as an error on deviceWapper. File "/config/custom_components/solis/platform2_portal.py", line 173, in _update_attributes _LOGGER.debug("Data received DEVICEWAPPER: %s", portaldata['deviceWapper']) KeyError: 'deviceWapper'

I think this is what you are looking for:

"realTimeDataBMS": [ { "baseUnit": "V", "key": "2tt", "name": "BMS voltage", "unit": "V", "value": "48.73" }, { "baseUnit": "A", "key": "2tu", "name": "BMS current", "unit": "A", "value": "6.70" }, { "baseUnit": "A", "key": "2tv", "name": "BMS limited charge current", "unit": "A", "value": "37.00" }, { "baseUnit": "A", "key": "2tw", "name": "BMS limited discharge current", "unit": "A", "value": "37.00" } ], "realTimeDataBattery": [ { "key": "1ff", "name": "Battery Status", "value": "Discharge" }, { "key": "1cr", "name": "Battery Voltage", "unit": "V", "value": "48.70" }, { "key": "1cs", "name": "Battery Current", "unit": "A", "value": "6.40" }, { "key": "1ct", "name": "Battery Power", "unit": "W", "value": "310" }, { "key": "1cv", "name": "Remaining Battery Capacity (SoC)", "unit": "%", "value": "28" }, { "key": "2tm", "name": "SOH", "value": "100" }, { "key": "1cx", "name": "Total Energy Charged", "unit": "kWh", "value": "19" }, { "key": "1cy", "name": "Total Energy Discharged", "unit": "kWh", "value": "23" }, { "key": "2vc", "name": "Energy Charged yesterday", "unit": "kWh", "value": "1.10" }, { "key": "2vd", "name": "Energy Discharged yesterday", "unit": "kWh", "value": "2.10" }, { "key": "1cz", "name": "Daily Energy Charged", "unit": "kWh", "value": "1.90" }, { "key": "1da", "name": "Daily Energy Discharged", "unit": "kWh", "value": "1.80" }, { "key": "1db", "name": "Monthly Energy Charged", "unit": "kWh", "value": "18.99" }, { "key": "1dc", "name": "Monthly Energy Discharged", "unit": "kWh", "value": "23.20" }, { "key": "1dd", "name": "Annual Energy Charged", "unit": "kWh", "value": "18.99" }, { "key": "1de", "name": "Annual Energy Discharged", "unit": "kWh", "value": "23.20" } ], "realTimeDataBypass": [ { "baseUnit": "V", "key": "2ve", "name": "Back-up AC voltage", "unit": "V", "value": "243.10" }, { "baseUnit": "W", "key": "2vg", "name": "Back-up load power", "unit": "W", "value": "10" } ], "realTimeDataEPMManage": [], "realTimeDataElect": [ { "key": "1fe", "name": "Power Grid Status", "value": "Buy Energy" }, { "baseUnit": "V", "key": "1bj", "name": "Power Grid Voltage R/U/A", "unit": "V", "value": "243.20" }, { "baseUnit": "A", "key": "1bm", "name": "Power Grid Current R/U/A", "unit": "A", "value": "2.98" }, { "baseUnit": "W", "key": "1bq", "name": "Power Grid Total Power", "unit": "W", "value": "0" }, { "baseUnit": "Var", "key": "1br", "name": "Power Grid Total Reactive Power", "unit": "Var", "value": "0" }, { "baseUnit": "VA", "key": "1bs", "name": "Power Grid Total Apparent Power", "unit": "VA", "value": "230" }, { "key": "1bt", "name": "Grid Power Factor", "value": "1.00" }, { "baseUnit": "kWh", "key": "1bu", "name": "Total On-grid Generation", "unit": "kWh", "value": "4" }, { "baseUnit": "kWh", "key": "1bv", "name": "Total Energy Purchased", "unit": "kWh", "value": "113" }, { "key": "1bw", "name": "Daily On-grid Energy", "unit": "kWh", "value": "0.00" }, { "key": "1bx", "name": "Daily Energy Purchased", "unit": "kWh", "value": "1.20" }, { "key": "1by", "name": "Monthly On-grid Energy", "unit": "kWh", "value": "121.7" }, { "key": "1bz", "name": "Monthly Energy Purchased", "unit": "kWh", "value": "206.70" }, { "key": "1ca", "name": "Annual On-grid Energy", "unit": "kWh", "value": "121.7" }, { "key": "1cb", "name": "Annual Energy Purchased", "unit": "kWh", "value": "206.70" }, { "baseUnit": "W", "key": "2tp", "name": "Meter power", "unit": "W", "value": "-1" } ], "realTimeDataElectricMeter": [], "realTimeDataImp": [ { "baseUnit": "V", "key": "1a", "name": "DC Voltage PV1", "unit": "V", "value": "4.60" }, { "baseUnit": "V", "key": "1b", "name": "DC Voltage PV2", "unit": "V", "value": "5.40" }, { "baseUnit": "A", "key": "1j", "name": "DC Current1", "unit": "A", "value": "0.00" }, { "baseUnit": "A", "key": "1k", "name": "DC Current2", "unit": "A", "value": "0.00" }, { "baseUnit": "V", "key": "1af", "name": "AC Voltage R/U/A", "unit": "V", "value": "243.10" }, { "baseUnit": "V", "key": "1ag", "name": "AC Voltage S/V/B", "unit": "V", "value": "0.00" }, { "baseUnit": "V", "key": "1ah", "name": "AC Voltage T/W/C", "unit": "V", "value": "0.00" }, { "baseUnit": "A", "key": "1ai", "name": "AC Current R/U/A", "unit": "A", "value": "0.90" }, { "baseUnit": "A", "key": "1aj", "name": "AC Current S/V/B", "unit": "A", "value": "0.00" }, { "baseUnit": "A", "key": "1ak", "name": "AC Current T/W/C", "unit": "A", "value": "0.00" }, { "baseUnit": "W", "key": "1ao", "name": "AC Output Total Power (Active)", "unit": "W", "value": "0" }, { "baseUnit": "Hz", "key": "1ar", "name": "AC Output Frequency R", "unit": "Hz", "value": "49.98" }, { "key": "1bd", "name": "Daily Generation (Active)", "unit": "kWh", "value": "2.60" }, { "key": "1be", "name": "Monthly Generation (Active)", "unit": "kWh", "value": "37.00" }, { "key": "1bf", "name": "Annual Generation (Active)", "unit": "kWh", "value": "37.0" }, { "key": "1bc", "name": "Total Generation (Active)", "unit": "kWh", "value": "37.00" }, { "key": "1df", "name": "Inverter Temperature", "unit": "℃", "value": "25.20" } ], "realTimeDataOther": [ { "key": "1vw", "name": "Year", "value": "21" }, { "key": "1vx", "name": "Month", "value": "10" }, { "key": "1vy", "name": "Day", "value": "28" }, { "key": "1wa", "name": "Minute", "value": "9" }, { "key": "1wb", "name": "Second", "value": "21" } ], "realTimeDataPower": [ { "baseUnit": "V", "key": "1a", "name": "DC Voltage PV1", "unit": "V", "value": "4.60" }, { "baseUnit": "V", "key": "1b", "name": "DC Voltage PV2", "unit": "V", "value": "5.40" }, { "baseUnit": "A", "key": "1j", "name": "DC Current1", "unit": "A", "value": "0.00" }, { "baseUnit": "A", "key": "1k", "name": "DC Current2", "unit": "A", "value": "0.00" }, { "baseUnit": "V", "key": "1af", "name": "AC Voltage R/U/A", "unit": "V", "value": "243.10" }, { "baseUnit": "V", "key": "1ag", "name": "AC Voltage S/V/B", "unit": "V", "value": "0.00" }, { "baseUnit": "V", "key": "1ah", "name": "AC Voltage T/W/C", "unit": "V", "value": "0.00" }, { "baseUnit": "A", "key": "1ai", "name": "AC Current R/U/A", "unit": "A", "value": "0.90" }, { "baseUnit": "A", "key": "1aj", "name": "AC Current S/V/B", "unit": "A", "value": "0.00" }, { "baseUnit": "A", "key": "1ak", "name": "AC Current T/W/C", "unit": "A", "value": "0.00" }, { "baseUnit": "W", "key": "1ao", "name": "AC Output Total Power (Active)", "unit": "W", "value": "0" }, { "baseUnit": "Hz", "key": "1ar", "name": "AC Output Frequency R", "unit": "Hz", "value": "49.98" }, { "baseUnit": "kWh", "key": "1aw", "name": "Generation Yesterday", "unit": "kWh", "value": "1.80" }, { "baseUnit": "kWh", "key": "1ru", "name": "Generation of Last Month (Active)", "unit": "kWh", "value": "0" }, { "key": "1bd", "name": "Daily Generation (Active)", "unit": "kWh", "value": "2.60" }, { "key": "1be", "name": "Monthly Generation (Active)", "unit": "kWh", "value": "37.00" }, { "key": "1bf", "name": "Annual Generation (Active)", "unit": "kWh", "value": "37.0" }, { "key": "1bc", "name": "Total Generation (Active)", "unit": "kWh", "value": "37.00" }, { "key": "1s", "name": "DC Power PV1", "unit": "W", "value": "0.0" }, { "key": "1t", "name": "DC Power PV2", "unit": "W", "value": "0.0" } ], "realTimeDataSetamEngine": [], "realTimeDataState": [], "realTimeDataTemp": [], "realTimeDataUPS": [], "realTimeDataUseElect": [ { "key": "1cj", "name": "Total Consumption Power ", "unit": "W", "value": "300.0" }, { "key": "1cn", "name": "Total Consumption Energy ", "unit": "kWh", "value": "53" }, { "key": "2vb", "name": "Consumption Energy Yesterday", "unit": "kWh", "value": "6.70" }, { "key": "1co", "name": "Daily Energy Used", "unit": "kWh", "value": "3.70" }, { "key": "1cp", "name": "Monthly Energy Used", "unit": "kWh", "value": "53.1" }, { "key": "1cq", "name": "Annual Energy Used", "unit": "kWh", "value": "53.1" } ], "receiveTimestamps": 1635436962000, "sensor": "0505", "signalQuality": "ico-wifi-5", "sn": "REMOVED", "state": 1, "updateDate": 1635444614000 }, "url": "//file.solarman.cn/", "deviceWapperInfo": {

[hultenvp]

Weird, you seem to have a previous version of the code. I changed the line to log portaldata instead of data before I reached out to you that I had a version ready, but you do not seem to have gotten those changes.

Still I think we have what we need:

{ "key": "1bw", "name": "Daily On-grid Energy", "unit": "kWh", "value": "0.00" }, { "key": "1bx", "name": "Daily Energy Purchased", "unit": "kWh", "value": "1.20 { "key": "1co", "name": "Daily Energy Used", "unit": "kWh", "value": "3.70" } I believe those two were the ones you'd like to have added for grid?

Cheers

[RCGrahamWilmott]

Yes, please. Not sure if you would prefer to have me run another test or push as a new version to HACS?

(I'm not sure what other values may be useful to anyone else)

[hultenvp]

If you wouldn't mind a test before merging would be great. I made the necessary code changes in branch 23. https://github.com/hultenvp/solis-sensor/tree/23-add-more-sensors-for-battery

Thanks, Peter

[hultenvp]

One question to verify I understand what the values mean, so I can improve the descriptions a bit. I particular the first one seems a bit ambiguous for non-native English speakers: Daily On-grid Energy: Energy given back to grid Daily Energy Purchased: Energy taken from grid Daily Energy Used: Total energy consumed by the household

Cheers

[RCGrahamWilmott]

I've been using the current version and it seems to be working perfectly. (I've checked the data matches the data on the website, which as expected it does).

sensor.solar_daily_grid_energy_used - seems to be the total amount of energy that has gone from the invertor (solar or battery) to the house. I think "total energy used by the house" would be a good summary.

sensor.solar_daily_grid_energy_purchased - This seems to be the amount of energy I purchased yesterday. (It isn't matching the amount of energy I am using today, but does match (but rounded down) to what I used yesterday). I am going to keep an eye on this...

sensor.solar_daily_on_grid_energy - I'm not currently exporting so can't check this. (Unless we get a very sunny day, I might not export much until the spring). However, I believe this to be the energy given back / sold to the grid. I think it would be great to rename this. I might see if I can "force" myself to export something and triple-check this is all working, but I am seeing zero in HA (as expected), and this was the correct variable on the https://m.ginlong.com/ website when I was able to export a bit.

Also, every icon in home assistant is the standard electricity icon. Prehaps some could be something like mdiHomeImportOutline / mdiHomeExportOutline / mdiTransmissionTowerImport / mdiTransmissionTowerExport / mdiBatteryPlus / mdiBatteryArrowUp / mdiBatteryMinus / mdiBatteryArrowDown to help make it clear what each sensor does?

[RCGrahamWilmott]

(Typo on line 366 of "/config/custom_components/solis/platform2_portal.py" - GRID_DTOTAL_POWER should be GRID_TOTAL_POWER )

[hultenvp]

Hi @RCGrahamWilmott ,

Thanks for the clarification. I have updated the readme a bit, we can change it further if you have progressing insights. I've also updated the icons following your suggestions. If you want to improve them further then feel free to make a pull request.

Typo fixed!

[hultenvp]

Hi @RCGrahamWilmott ,

I saw you reviewed my PR to get integrated into HACS. Didn't know you were involved in HACS, thanks for reviewing. Do you have any insight when the others will review? Is that an ad-hoc process or do they review and integrate periodically?

If unclear, do you know if I can continue to release on my project or does that interfere with the HACs integration PR? I assume there are no dependencies, but I'd like to be sure ;-)

TIA, Peter

[hultenvp]

Merged and release in v0.3.3. No default repo in HACS yet