Hypfer / deye-microinverter-cloud-free

Documentation and tooling to use Deye microinverters without the solarman cloud
118 stars 16 forks source link

Support for SUN-M80G3-EU-Q0 #14

Closed czoller closed 5 months ago

czoller commented 5 months ago

Hi, I'm running the Dummycloud with a Deye SUN-M80G3-EU-Q0 with Firmware MW3_16U_5406_2.32-D1 and I receive a lot of data packets that contain bullshit after being parsed. Most of the time, I receive multiple packets in row and often the first one contains reasonable data and the latter bullshit. But this is not always the case.

Here are some example logs:

 [2024-04-08T17:14:03.092Z] [TRACE] 2024-04-08T17:14:03.092Z Data received from client: �JB,��Tgi��e�2305107146Z�        ��
▒�▒�▒�▒�V0.2.0.1V0.1.1.7V1.2.1.8@'q2305107146-1    2305107146-2    2305107146-3    2305107146-4    �,
�&�<▒   ▒��,I
�&�=

[2024-04-08T17:14:03.092Z] [TRACE] 2024-04-08T17:14:03.092Z Data a54a01104215162c9ad6ea0108540516670069030000d60fad6501008e0500003233303531303731343604005a000000ef06000000000000100900000000000000000000921304000000d20a18fc18fc18fc18fc0000000000000000000000000f01000016010000000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000400040000000000000327000003710000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c1804081109010001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000020c15
[2024-04-08T17:14:03.092Z] [DEBUG] Received packet of type "DATA"
[2024-04-08T17:14:03.092Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 27.1, i: 0, w: 0, kWh_today: 0.4, kWh_total: 80.7 },
    '2': { v: 27.8, i: 0, w: 0, kWh_today: 0.4, kWh_total: 88.1 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 4,
    kWh_today: 0.9,
    kWh_total: 177.5,
    v: 232,
    i: 0,
    hz: 50.1
  },
  inverter: { radiator_temp_celsius: 27.7 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: 2024-04-08T17:09:01.000Z,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}
[2024-04-08T17:14:03.093Z] [DEBUG] Data event emitted.
[2024-04-08T17:14:03.093Z] [TRACE] Response a50a00101216162c9ad6ea0101db251466000000005a15
[2024-04-08T17:14:03.297Z] [TRACE] 2024-04-08T17:14:03.297Z Data received from client: �*B,���Tgi��e�3806571563-01230510714�"o��
                                                                                                                                ;
                                                                                                                                 ;&

                                                                                                                                   ��

,

        ����A   �#x@�,�.�0�
                           ���`���@��LSW5BLE_MW17_5406_RELAY_270_1.00-SH#
[2024-04-08T17:14:03.298Z] [DEBUG] Received packet of type "DATA"
[2024-04-08T17:14:03.298Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 0, i: 0, w: 0, kWh_today: 1, kWh_total: 195.5 },
    '2': { v: 0, i: 3339.1, w: 0, kWh_today: 1, kWh_total: 475 },
    '3': { v: 474.3, i: 25.6, w: 12142.08, kWh_today: 1, kWh_total: 253 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 231.4, kWh_total: 2 }
  },
  grid: {
    active_power_w: 0,
    kWh_today: 0.49,
    kWh_total: 89235128.2,
    v: 1387.6,
    i: 0.8,
    hz: 0
  },
  inverter: { radiator_temp_celsius: 0 },
  inverter_meta: {
    rated_power_w: 30,
    mppt_count: 0,
    startup_self_check_time: 0,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 0,
    grid_overfreq_load_reduction_percent: 256,
    grid_v_limit_upper: 0,
    grid_v_limit_lower: 0,
    grid_freq_hz_limit_upper: 0,
    grid_freq_hz_limit_lower: 0,
    protocol_ver: '\x00\x02\x00\x00\x00\x02\x0B;',
    dc_master_fw_ver: '\x0B;\x07&\x04\x0B\x04\x0B',
    ac_fw_ver: '\x14\x1E\x14\x1E\x12\x0E\x12\x0E'
  }
}
[2024-04-08T17:14:03.299Z] [DEBUG] Data event emitted.
[2024-04-08T17:14:03.299Z] [TRACE] Response a50a00101217172c9ad6ea0101db251466000000005c15

Can you make any sense of it?

Hypfer commented 5 months ago

Oh interesting. I was wondering if it would work with the new SUN-Mxx inverters or if they've changed something about the protocol. Turns out it's the latter. I was kinda expecting it to be this was though after having seen the Deye SUN-12K packets and those being entirely different as well.

The data you're receiving isn't garbage. The parser just doesn't know how to make sense of it.

What is interesting is this:

LSW5BLE_MW17_5406_RELAY_270_1.00-SH

Is your SUN-M80 one with the NA relay integrated? Afaik there were some units that required the external SUN-MI-RELAY-01 and others that come with it inside it. If so, this kinda looks a bit like as if the internal relay module to the cloud still pretends to be an external one or something like that.

There are a few ToDos here.

First of all, you should check if the data is actually correct and not just plausible. For that, either compare with what you see in the solarman cloud or check what you can read via modbus.

Then, if the hypothesis is correct, someone unfortunately would have to use the solarman cloud and see if they can see any data from that relay module. If yes, they'd then have to sample the data there while also doing a package capture so that one can make sense of the package structure by comparing values in it with known data at a given point in time.

If the data you see in the first packets is actually correct and there's nothing relevant in the relay packets, it might be enough to figure out a way to detect that they are relay packets and discard them. Looking at the kaitai struct I wrote for the packets again, maybe it's as easy as this field being a 03? https://github.com/Hypfer/deye-microinverter-cloud-free/blob/e35914b2d1f08cc1431ccd628f65427f390acc90/assets/deye_sun600.ksy#L43-L44 Can't verify that with the string representation in the log you posted though

czoller commented 5 months ago

Thanks for your answer, @Hypher. It's a bit annyoing that I can only test it during the day. I hope, I can try something out soon.

The relay is external.

czoller commented 5 months ago

First of all, you should check if the data is actually correct and not just plausible. For that, either compare with what you see in the solarman cloud or check what you can read via modbus.

For me the content of the first packet looks quite correct.

This is what I get here:

{
  "pv": {
    "1": {
      "v": 30.8,
      "i": 1.2,
      "w": 36.96,
      "kWh_today": 0,
      "kWh_total": 82
    },
    "2": {
      "v": 31.2,
      "i": 1.2,
      "w": 37.44,
      "kWh_today": 0,
      "kWh_total": 89.5
    },
    "3": {
      "v": 0,
      "i": 0,
      "w": 0,
      "kWh_today": 0,
      "kWh_total": 0
    },
    "4": {
      "v": 0,
      "i": 0,
      "w": 0,
      "kWh_today": 0,
      "kWh_total": 0
    }
  },
  "grid": {
    "active_power_w": 79,
    "kWh_today": 0,
    "kWh_total": 180.3,
    "v": 239,
    "i": 0.3,
    "hz": 50
  },
  "inverter": {
    "radiator_temp_celsius": 23.7
  },
  "inverter_meta": {
    "rated_power_w": 800,
    "mppt_count": 2,
    "startup_self_check_time": 60,
    "current_time": "2024-04-10T15:38:07.000Z",
    "grid_freq_hz_overfreq_load_reduction_starting_point": 50.2,
    "grid_overfreq_load_reduction_percent": 44,
    "grid_v_limit_upper": 275,
    "grid_v_limit_lower": 183,
    "grid_freq_hz_limit_upper": 51.5,
    "grid_freq_hz_limit_lower": 47.5,
    "protocol_ver": "V0.2.0.1",
    "dc_master_fw_ver": "V0.1.1.7",
    "ac_fw_ver": "V1.2.1.8"
  }
}

Compared to what jedie/inverter-connect> returns:

┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ Counter ┃                                            Name ┃ Value                    ┃ Register ┃ Length ┃             Raw data ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│       1 │                                     PV1 Voltage │         30.8 V           │   006D   │   1    │                 0134 │
│       2 │                                     PV2 Voltage │         31.2 V           │   006F   │   1    │                 0138 │
│       3 │                                     PV1 Current │          1.2 A           │   006E   │   1    │                 000c │
│       4 │                                     PV2 Current │          1.2 A           │   0070   │   1    │                 000c │
│       5 │                                Daily Production │          0.0 kWh         │   003C   │   1    │                 0000 │
│       6 │                              Daily Production 1 │          0.0 kWh         │   0041   │   1    │                 0000 │
│       7 │                              Daily Production 2 │          0.0 kWh         │   0042   │   1    │                 0000 │
│       8 │                                Total Production │         51.3 kWh         │   003F   │   2    │                 0201 │
│       9 │                              Total Production 1 │         82.0 kWh         │   0045   │   1    │                 0334 │
│      10 │                              Total Production 2 │         89.5 kWh         │   0047   │   1    │                 037f │
│      11 │                        Active Power Regulations │           73 %           │   0028   │   1    │                 0049 │
│      12 │                                      AC Voltage │        239.0 V           │   0049   │   1    │                 0956 │
│      13 │                                    Grid Current │          0.3 A           │   004C   │   1    │                 0003 │
│      14 │                             AC Output Frequency │         50.0 Hz          │   004F   │   1    │                 1388 │
│      15 │                          Grid Voltage Upp Limit │        275.0 V           │   001B   │   1    │                 0abe │
│      16 │                        Grid Voltage Lower Limit │        183.0 V           │   001C   │   1    │                 0726 │
│      17 │                      Grid Frequency Upper Limit │         51.5 Hz          │   001D   │   1    │                 141e │
│      18 │                      Grid Frequency Lower Limit │         47.5 Hz          │   001E   │   1    │                 128e │
│      19 │ Overfrequency And Load Reduction Starting Point │         50.2 Hz          │   0022   │   1    │                 139c │
│      20 │     Overfrequency And Load Reduction Percentage │           44 %           │   0023   │   1    │                 002c │
│      21 │                                   ON-OFF Enable │           ON             │   002B   │   1    │                 0001 │
│      22 │                        Island Protection Enable │      Enabled             │   002E   │   1    │                 0001 │
│      23 │              Overfrequency&Load-shedding Enable │      Enabled             │   0031   │   1    │                 0001 │
│      24 │                                  Running Status │        Fault             │   003B   │   1    │                 0004 │
│      25 │                  Total AC Output Power (Active) │         79.0 W           │   0056   │   2    │             03160000 │
│      26 │                            Radiator Temperature │         23.7 °C          │   005A   │   1    │                 0d2a │
│      27 │                                     Inverter ID │ ********************     │   0003   │   5    │ ******************** │
│      28 │                                Hardware Version │     v0.1.0.2             │   000C   │   1    │                 0102 │
│      29 │                      DC Master Firmware Version │     v0.1.1.7             │   000D   │   1    │                 0117 │
│      30 │                              AC Version. Number │     v1.2.1.8             │   000E   │   1    │                 1218 │
│      31 │                                     Rated Power │        800.0 W           │   0010   │   1    │                 1f40 │
│      32 │                  Communication Protocol Version │     v0.2.0.1             │   0012   │   1    │                 0201 │
│      33 │                     Start-up Self-checking Time │           60 s           │   0015   │   1    │                 003c │
│      34 │                                     Update Time │ <raw hex: 18040a0f2b0b>  │   0016   │   3    │         18040a0f2b0b │
│      35 │                               Soft Start Enable │     Disabled             │   002F   │   1    │                 0000 │
│      36 │                         Power Factor Regulation │         -4.3             │   0032   │   1    │                 ffd5 │
│      37 │                        Restore Factory Settings │     Disabled             │   0036   │   1    │                 0000 │
│      38 │                                       PV1 Power │        36.96 W           │    -     │   -    │           (Computed) │
│      39 │                                       PV2 Power │        37.44 W           │    -     │   -    │           (Computed) │
│      40 │                                     Total Power │         74.4 W           │    -     │   -    │           (Computed) │
└─────────┴─────────────────────────────────────────────────┴──────────────────────────┴──────────┴────────┴──────────────────────┘

The only difference is the total production, but here the one from dummycloud looks more reasonable.

The values also match with those in the web UI.

Hypfer commented 5 months ago

For debugging, I'd recommend adding some code here that either logs as hex or writes to a file so that you can share one of the packets that fail to parse properly https://github.com/Hypfer/deye-microinverter-cloud-free/blob/e35914b2d1f08cc1431ccd628f65427f390acc90/dummycloud/src/DummyCloud.js#L31

czoller commented 5 months ago

I'm sorry, it looks like I had removed that line by accident for the second packet in my first post. Here is one group of packets with HEX and JSON:

[2024-04-10T15:43:10.935Z] [TRACE] 2024-04-10T15:43:10.935Z Data a54a01104231322c9ad6ea0108541d75680049100000723eae650100cf060000323330353130373134360400000000000b0700000000000056090000000003000000000088134f000000420918fc18fc18fc18fc00000000000000000000000034010c0038010c00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f40020100040000000000000000000003340000037f0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c18040a0f26070001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000020815
[2024-04-10T15:43:10.936Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 30.8, i: 1.2, w: 36.96, kWh_today: 0, kWh_total: 82 },
    '2': { v: 31.2, i: 1.2, w: 37.44, kWh_today: 0, kWh_total: 89.5 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 79,
    kWh_today: 0,
    kWh_total: 180.3,
    v: 239,
    i: 0.3,
    hz: 50
  },
  inverter: { radiator_temp_celsius: 23.7 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: 2024-04-10T15:38:07.000Z,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:11.046Z] [TRACE] 2024-04-10T15:43:11.046Z Data a54a01104232332c9ad6ea810854025c680060210000000000000100b70600003233303531303731343604005401000008070000000000005609000000000700000000008813b4000000640a18fc18fc18fc18fc00000000000000000000000040011a003c011d00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001100000000000003330000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000029b15
[2024-04-10T15:43:11.046Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 32, i: 2.6, w: 83.2, kWh_today: 1.5, kWh_total: 81.9 },
    '2': { v: 31.6, i: 2.9, w: 91.64, kWh_today: 1.7, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 180,
    kWh_today: 3.4,
    kWh_total: 180,
    v: 239,
    i: 0.7,
    hz: 50
  },
  inverter: { radiator_temp_celsius: 26.6 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:11.118Z] [TRACE] 2024-04-10T15:43:11.118Z Data a54a01104233342c9ad6ea810854d55a680033200000000000000100b60600003233303531303731343604005401000008070000000000004c09000000000700000000009213b6000000640a18fc18fc18fc18fc0000000000000000000000004d011a0040011d00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001100000000000003330000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000025215
[2024-04-10T15:43:11.119Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 33.3, i: 2.6, w: 86.58, kWh_today: 1.5, kWh_total: 81.9 },
    '2': { v: 32, i: 2.9, w: 92.8, kWh_today: 1.7, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 182,
    kWh_today: 3.4,
    kWh_total: 180,
    v: 238,
    i: 0.7,
    hz: 50.1
  },
  inverter: { radiator_temp_celsius: 26.6 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:11.190Z] [TRACE] 2024-04-10T15:43:11.190Z Data a54a01104234352c9ad6ea810854a8596800061f0000000000000100b50600003233303531303731343604005401000008070000000000006009000000000700000000008813b90000005a0a18fc18fc18fc18fc0000000000000000000000004d011a0045011d00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d000000000000000000000002fd15
[2024-04-10T15:43:11.191Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 33.3, i: 2.6, w: 86.58, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 32.5, i: 2.9, w: 94.25, kWh_today: 1.6, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 185,
    kWh_today: 3.4,
    kWh_total: 180,
    v: 240,
    i: 0.7,
    hz: 50
  },
  inverter: { radiator_temp_celsius: 26.5 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:11.264Z] [TRACE] 2024-04-10T15:43:11.264Z Data a54a01104235362c9ad6ea8108547a586800d81d0000000000000100b40600003233303531303731343604004a01000007070000000000007409000000000800000000008813cf0000003c0a18fc18fc18fc18fc00000000000000000000000041011f003e012100000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000029715
[2024-04-10T15:43:11.265Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 32.1, i: 3.1, w: 99.51, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 31.8, i: 3.3, w: 104.94, kWh_today: 1.6, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 207,
    kWh_today: 3.3,
    kWh_total: 179.9,
    v: 242,
    i: 0.8,
    hz: 50
  },
  inverter: { radiator_temp_celsius: 26.2 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:11.327Z] [TRACE] 2024-04-10T15:43:11.327Z Data a54a01104236372c9ad6ea8108544d576800ab1c0000000000000100b30600003233303531303731343604004a01000007070000000000006a09000000000800000000009213cb000000320a18fc18fc18fc18fc00000000000000000000000041011e004c011f00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000023915
[2024-04-10T15:43:11.327Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 32.1, i: 3, w: 96.3, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 33.2, i: 3.1, w: 102.92, kWh_today: 1.6, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 203,
    kWh_today: 3.3,
    kWh_total: 179.9,
    v: 241,
    i: 0.8,
    hz: 50.1
  },
  inverter: { radiator_temp_celsius: 26.1 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:11.392Z] [TRACE] 2024-04-10T15:43:11.392Z Data a54a01104237382c9ad6ea810854205668007e1b0000000000000100b20600003233303531303731343604004a01000007070000000000007e09000000000700000000008813c0000000f00a18fc18fc18fc18fc00000000000000000000000037011d003c011f00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000027f15
[2024-04-10T15:43:11.392Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 31.1, i: 2.9, w: 90.19, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 31.6, i: 3.1, w: 97.96, kWh_today: 1.6, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 192,
    kWh_today: 3.3,
    kWh_total: 179.9,
    v: 243,
    i: 0.7,
    hz: 50
  },
  inverter: { radiator_temp_celsius: 28 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:12.555Z] [TRACE] 2024-04-10T15:43:12.555Z Data a54a01104238392c9ad6ea810854f3546800501a0000000000000100b10600003233303531303731343604004a01000007070000000000007e09000000000800000000008813d1000000e60a18fc18fc18fc18fc00000000000000000000000045011f004a012000000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000024915
[2024-04-10T15:43:12.556Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 32.5, i: 3.1, w: 100.75, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 33, i: 3.2, w: 105.6, kWh_today: 1.6, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 209,
    kWh_today: 3.3,
    kWh_total: 179.9,
    v: 243,
    i: 0.8,
    hz: 50
  },
  inverter: { radiator_temp_celsius: 27.9 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:12.615Z] [TRACE] 2024-04-10T15:43:12.614Z Data a54a011042393a2c9ad6ea810854c653680023190000000000000100b00600003233303531303731343604004a01000007070000000000005609000000000b0000000000921315010000aa0a18fc18fc18fc18fc00000000000000000000000040012a0042012c00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037d0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d000000000000000000000002e615
[2024-04-10T15:43:12.615Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 32, i: 4.2, w: 134.4, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 32.2, i: 4.4, w: 141.68, kWh_today: 1.6, kWh_total: 89.3 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 277,
    kWh_today: 3.3,
    kWh_total: 179.9,
    v: 239,
    i: 1.1,
    hz: 50.1
  },
  inverter: { radiator_temp_celsius: 27.3 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:12.677Z] [TRACE] 2024-04-10T15:43:12.677Z Data a54a0110423a3b2c9ad6ea81085498526800f5170000000000000100af0600003233303531303731343604004a01000007070000000000006009000000000c000000000092132c010000be0a18fc18fc18fc18fc00000000000000000000000048012d0042012f00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037c0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d000000000000000000000002cb15
[2024-04-10T15:43:12.677Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 32.8, i: 4.5, w: 147.6, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 32.2, i: 4.7, w: 151.34, kWh_today: 1.6, kWh_total: 89.2 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 300,
    kWh_today: 3.3,
    kWh_total: 179.9,
    v: 240,
    i: 1.2,
    hz: 50.1
  },
  inverter: { radiator_temp_celsius: 27.5 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:13.976Z] [TRACE] 2024-04-10T15:43:13.976Z Data a54a0110423b3c2c9ad6ea8108544d516800aa160000000000000100ae0600003233303531303731343604004001000006070000000000007e09000000000900000000008813ec000000dc0a18fc18fc18fc18fc00000000000000000000000073011f0036012600000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f4002010004000f001000000000000003320000037c0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c3000000000000001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000021f15
[2024-04-10T15:43:13.977Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 37.1, i: 3.1, w: 115.01, kWh_today: 1.5, kWh_total: 81.8 },
    '2': { v: 31, i: 3.8, w: 117.8, kWh_today: 1.6, kWh_total: 89.2 },
    '3': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 0, kWh_total: 0 }
  },
  grid: {
    active_power_w: 236,
    kWh_today: 3.2,
    kWh_total: 179.8,
    v: 243,
    i: 0.9,
    hz: 50
  },
  inverter: { radiator_temp_celsius: 27.8 },
  inverter_meta: {
    rated_power_w: 800,
    mppt_count: 2,
    startup_self_check_time: 60,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 50.2,
    grid_overfreq_load_reduction_percent: 44,
    grid_v_limit_upper: 275,
    grid_v_limit_lower: 183,
    grid_freq_hz_limit_upper: 51.5,
    grid_freq_hz_limit_lower: 47.5,
    protocol_ver: 'V0.2.0.1',
    dc_master_fw_ver: 'V0.1.1.7',
    ac_fw_ver: 'V1.2.1.8'
  }
}

[2024-04-10T15:43:14.122Z] [TRACE] 2024-04-10T15:43:14.122Z Data a52a0110423d3e2c9ad6ea011354207568004d100000723eae65aa0000000000333830363537313536332d3031000000323330353130373134360000540808000001010100000000000000000000000000000000000000000201c02200000000000000000000e79c87120001000000000002000000020b3b0b3b0726040b040b141e141e128e128e000a000a012c001e001e000a000a000a000a09d807a31392128e004109e200230014007800140000000100000000000000031f4005dc2cec2ee030d40bb80096ea60138800fa000000010000000000000000094b09441381000402fdfd6800000000000001000000000000000000000000000000000000000000000000000000000000004c535735424c455f4d5731375f353430365f52454c41595f3237305f312e30302d53480000000000002615
[2024-04-10T15:43:14.122Z] [DEBUG] DATA packet data from ::ffff:192.168.179.10 {
  pv: {
    '1': { v: 0, i: 0, w: 0, kWh_today: 1, kWh_total: 195.5 },
    '2': { v: 0, i: 4016.7, w: 0, kWh_today: 1, kWh_total: 475 },
    '3': { v: 474.3, i: 25.6, w: 12142.08, kWh_today: 1, kWh_total: 253 },
    '4': { v: 0, i: 0, w: 0, kWh_today: 231.4, kWh_total: 2 }
  },
  grid: {
    active_power_w: 0,
    kWh_today: 0.49,
    kWh_total: 89235128.2,
    v: 1387.6,
    i: 0.8,
    hz: 0
  },
  inverter: { radiator_temp_celsius: 0 },
  inverter_meta: {
    rated_power_w: 30,
    mppt_count: 0,
    startup_self_check_time: 0,
    current_time: Invalid Date,
    grid_freq_hz_overfreq_load_reduction_starting_point: 0,
    grid_overfreq_load_reduction_percent: 256,
    grid_v_limit_upper: 0,
    grid_v_limit_lower: 0,
    grid_freq_hz_limit_upper: 0,
    grid_freq_hz_limit_lower: 0,
    protocol_ver: '\x00\x02\x00\x00\x00\x02\x0B;',
    dc_master_fw_ver: '\x0B;\x07&\x04\x0B\x04\x0B',
    ac_fw_ver: '\x14\x1E\x14\x1E\x12\x0E\x12\x0E'
  }
}
czoller commented 5 months ago

For me, the easiest way for now is to just discard those packets, that don't have a valid date, after parsing. But that might not be the proper solution that you have in mind...

Hypfer commented 5 months ago

Hm. Those data packets that get parsed correct-ish looking have what I called payload_magic in the kaitai struct set to 129 as opposee to the expected 1. Maybe that should be read as 0b10000001 and 0b00000001 respectively and that bit is a flag?

It looks like it's data from a prior point in time. But why would the inverter send that to the cloud?

Wild guess but maybe to combat that there are holes in their graphs if there was a short internet outage and therefore the logger now keeps n to n-10 packets and always sends them on each report interval?

Hypfer commented 5 months ago

Please try https://github.com/Hypfer/deye-microinverter-cloud-free/commit/b29b255268695acc0fe1ff0c50704cb4d9470d84

czoller commented 5 months ago

Thanks a lot. Tomorrow, I'm not at home during the day. I'm gonna try it on Friday.

czoller commented 5 months ago

Please try b29b255

Looks good.

But, is it normal that the timestamps in inverter_meta.current_time doesn't correspond to the actual current time but to the time of the previous round?

Actual Time inverter_meta.current_time
2024-04-12T16:08:31.010Z 2024-04-12T16:07:56.000Z
2024-04-12T16:13:34.114Z 2024-04-12T16:08:31.000Z
2024-04-12T16:18:38.143Z 2024-04-12T16:13:34.000Z
2024-04-12T16:23:41.551Z 2024-04-12T16:18:38.000Z
2024-04-12T16:28:49.474Z 2024-04-12T16:23:42.000Z
2024-04-12T16:38:53.446Z 2024-04-12T16:28:49.000Z
Hypfer commented 5 months ago

If you take the first packet from the log you've posted

[2024-04-10T15:43:10.935Z] [TRACE] 2024-04-10T15:43:10.935Z Data a54a01104231322c9ad6ea0108541d75680049100000723eae650100cf060000323330353130373134360400000000000b0700000000000056090000000003000000000088134f000000420918fc18fc18fc18fc00000000000000000000000034010c0038010c00000000000000000056302e322e302e3156302e312e312e3756312e322e312e38000000001f40020100040000000000000000000003340000037f0000000000000000323330353130373134362d3120202020323330353130373134362d3220202020323330353130373134362d3320202020323330353130373134362d3420202020139c002c00010000000000000abe0726141e128e003c18040a0f26070001000102011712180000000000000000000000000000000000ff000000000000139c002c00000000000000000049000000000001000000000abe0726141e128e7f3d0000000000000000000000020815

and decode that, you'll find that the date contained is 18 04 0a 0f 26 07 which is 24, 4, 10, 15, 38, 7 in decimal meaning 2024-04-10 15:38:07

So yeah, that's actually what the logger reports. I'd suspect that the timestamp is the start of the 5 minute sampling period over which the data is averaged.

czoller commented 5 months ago

If you take the first packet from the log you've posted

[2024-04-10T15:43:10.935Z] [TRACE] 2024-04-10T15:43:10.935Z Data a54a...

and decode that, you'll find that the date contained is 18 04 0a 0f 26 07 which is 24, 4, 10, 15, 38, 7 in decimal meaning 2024-04-10 15:38:07

Yes, I had no doubt that the parser is correct, I was just wondering if that's the normal behavior of these inverters.

So yeah, that's actually what the logger reports. I'd suspect that the timestamp is the start of the 5 minute sampling period over which the data is averaged.

Or the inverter has no internal wall clock at all and just stores and passes the last timestamp received from the cloud (which leads to the same effect in the end).

Hypfer commented 5 months ago

Or the inverter has no internal wall clock at all and just stores and passes the last timestamp received from the cloud (which leads to the same effect in the end).

Sure that is possible, but what would be the point of doing that?

czoller commented 5 months ago

Of doing what? Having no internal wall clock or storing and passing the last timestamp?

czoller commented 5 months ago

@Hypfer : Thanks a lot for your help and your whole work on this project!