[Request] Setting Netprofile function

Kraego commented 1 year ago

Is your feature request related to a problem? Please describe.

is it possible to configure the netprofile over opendtu, to AT (like described here: https://e4mobility.at/blog/hoymiles-netzprofil-richtig-einstellen)?

Describe the solution you'd like

Set the profile like described here: https://e4mobility.at/blog/hoymiles-netzprofil-richtig-einstellen

Describe alternatives you've considered

None except buying a DTU-Wlite just to configure the profile

Additional context

No response

androdlang commented 1 year ago

I confirm the need of setting the netprofile for some network providers (Salzburg-Netz in my case), otherwise without setting the profile the infeed is not accepted. Luckily I have a DTU-Lite, but I cannot accept buying this stick for configuring only. So it would be a great enhancement, thank you in advance.

stefan123t commented 8 months ago

We have been able to extract one/two Grid Profiles from a mock DTU, which allowed us to decode about 90% of the data.

Though we still lack proper traces for applying such a grid profile using the DTU W-Lite or DTU Pro in order to implement the necessary Update Grid Profile command in OpenDTU / Ahoy DTU.

If you have access to an DTU W-Lite / DTU Pro and are willing to provide the necessary trace data we can progress this issue.

stefan123t commented 8 months ago

Since the Grid Profile can be understood and read we can now take the next step and apply a modified Grid Profile. See this https://github.com/lumapu/ahoy/issues/365#issuecomment-1823596603 for the next steps to try applying a new / changed Grid Profile to one of the inverters using the respective DOWN_PRO Command.

Wie wird das DOWN_PRO (0x0E) / DOWN_DAT (0x0A) verwendet

Type_Init (0xFF=255) Das SubCmd = Type_Init (0xFF) scheint nur bei MainCmd = DOWN_PRO (0x0E=14) oder DOWN_DAT (0x0A=10) verwendet zu werden.

stefan123t commented 7 months ago

@tbnobody, appreciate all the documentation you added in tbnobody/OpenDTU@c9508d2. I wonder how to send a Command with a payload which is longer than the usual 32 bytes from the DTU to the inverter ?

To install the country specific Grid Profile on the inverter we have the following documented so far: https://github.com/lumapu/ahoy/wiki/Protocol#wie-wird-das-down_pro-0x0e--down_dat-0x0a-verwendet

The currently known Grid Profiles in Hex/Binary Format have already been documented in the Wiki https://github.com/tbnobody/OpenDTU/wiki/Grid-Profile-Parser Especially when we would like to send a a Grid Profile (e.g. the austrian AT_TOR_Erzeuger_Default seems to be a very common request) we need to use a "MultiRequest" Command to send a Grid Profile using DOWN_PRO 0x0E and INIT 0xFF

An example script for the (here AhoyDTU) Web API would look something like the following prseudo-python script to use such a Multi Request Command for DOWN_PRO / INIT and the AT-TOR Grid Profile as a payload.

espUrl="http://192.168.4.1/api"

def setGrid(iv):
    r = requests.post(espUrl + "/ctrl", json={
        "id": iv,
        "cmd": 0x0E, // DOWN_PRO
        "val": 0xFF, // INIT
        "payload": 0x0c 00 20 00 00 0b 08 fc 07 30 00 0f 09 f9 00 01 02 3f 00 32 0a 55 00 08 09 f9 10 00 13 88 12 8e 00 01 14 1e 00 01 20 00 00 01 30 03 02 58 09 cb 07 a3 13 92 12 8e 40 00 07 d0 00 10 50 00 00 01 13 9c 01 90 00 32 60 04 00 01 09 e2 0a 10 08 7e 80 00 00 00 08 44 01 2c 08 a0 09 6f 09 b4 01 2c 90 00 00 00 00 5f b0 00 00 00 01 f4 00 5f 70 02 00 01 27 10 a0 02 00 00 00 00 6e 8b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // remove the spaces to get a single Hex/Binary block
    })
    if r.status_code == 200:
        obj = r.json()
        print(obj)
    else:
        print(f"error http: {r.status_code}")

getGrid(0)
import requests
import json

So the procedure is known in principle, but instead of a MultiResponse / MultiData Command (e.g. DevInfo Status, Alarm, etc), we also need some kind of MultiRequest Commands which allow us to send a larger Payload. I.e. at least larger than the max 32 Bytes of 1 frame.

I do not know if this is currently implemented or what information is missing for you to implement that pre-requisite for setting a Grid Profile ?

stefan123t commented 7 months ago

@tbnobody currently the Grid Profile Parser is almost feature complete! Hurray :tada: ! Only positive Leading vs. negative Lagging Power Factor is missing ;)

The next step would be the implementation to send one of the about twelve known / even a custom Grid Profile to the inverter using MultiFrame Request Commands as detailled using DOWN_PRO 0x0e and INIT 0xFF above.

Regarding the customization of Grid Profiles, we would need the upper and lower limit / ranges for all the given values for input form validation added to our Parser / Grid Profile data. Also note that that some of the values are fixed, but most can be changed within those ranges given. If no range is given ~ usually the value cannot be changed.

I have already documented some of the Grid Profiles with Range information here:

Especially the binary ... Function Activated options would make for an easy "first target" to enable / disable some of the Inverters Grid Profile sections.

It would therefor also be nice and logical to show some (o ) disabled and ( *) enabled icons within the (initially) folded section headers to directly spot the sections currently active / deactivated within the folds.

For purposes of copy & paste and/or printing I would also appreciate if the complete Grid Profile could be unfolded, currently the other sections are automatically collapsed if I open another section. Though the Support the development section with the binary / hex-formatted Grid Profile should remain for debug and support purposes.

Find below an illustrative image of the most common Grid Profile (>70% of known inverters use it, currently 84 entries out of a total of 118 Grid Profile entries on our Wiki):

stefan123t commented 6 months ago

Prima! Danke an nivadis für den Dump beim Grid Profile DOWN_DAT 0x0A Kommando:

Ich glaube das ist es:

TX 6.316844960000000,0x7E 0A 80722536 74438179 01 03002000000A08FC0730001E0B3B0001 E0 7F,,
TX 6.366399360000000,0x7E 0A 80722536 74438179 02 040B001E09E210001388128E0001141E C0 7F,,
TX 6.417439200000000,0x7E 0A 80722536 74438179 03 000120000001300302580AC807A31392 89 7F,,
TX 6.467220000000000,0x7E 0A 80722536 74438179 04 128E400007D003E850080001139C0190 87 7F,,
TX 6.517172800000000,0x7E 0A 80722536 74438179 05 001001F6137470020001271080000000 65 7F,,
TX 6.567292640000000,0x7E 0A 80722536 74438179 06 085B012C08B70941099D012C90000000 82 7F,,
TX 6.628907360000000,0x7E 0A 80722536 74438179 07 FFA1B000000001F4005FA00200000000 C5 7F,,
TX 6.667958880000000,0x7E 0A 80722536 74438179 88 9C27 17 7F,,

TX 6.767887520000000,0x7E 0A 80722536 74438179 88 9C27 17 7F,,
... alle 100 ms wiederholt
TX 8.984026399999999,0x7E 0A 80722536 74438179 88 9C27 17 7F,,
RX 9.041319200000000,0x7E 8A 80722536 74438179 81 000003002000 9F19 80 7F,,
TX 9.107654080000000,0x7E 0A 80722536 74438179 88 9C27 17 7F,,
RX 9.113543840000000,0x7E 8A 80722536 74438179 81 000003002000 9F19 80 7F,,
... alle 100 ms wiederholt
TX 26.587171999999999,0x7E 0A 80722536 74438179 88 9C27 17 7F,,
RX 26.589525120000001,0x7E 8A 80722536 74438179 81 000003002000 9F19 80 7F,,
... die CRC16 Checksumme wird bis zu 20 Sekunden wiederholt
... dann sollte der Inverter das persistiert haben.

... kurze Pause ~120-130 ms
TX 26.717758400000001,0x7E 15 80722536 74438179 80 0B0065A9393B000000000000000039F6B1 7F,,
RX 26.724774239999999,0x7E 95 80722536 74438179 01 000100F9012C02E9000001A0000A099FB9 7F,,

Man sieht in der letzten TX Zeile des oberen Blocks die CRC16 über das Grid Profile. Danach wiederholt die DTU brav das letzte Paket 0x88 mit der Checksumme 0x9C27 bis 26.587 Sekunden, also insgesamt ca. 20 Sekunden lang, das sollte genügen, damit der WR das Grid Profile persistieren / flashen kann.

Das o.a. Grid Profile ist Nummer 6 in der entsprechenden Excel Liste der bisher dokumentierten Grid Profile: https://github.com/tbnobody/OpenDTU/issues/1606#issuecomment-1889104635

03 00 20 00 00 0a 08 fc 07 30 00 1e 0b 3b 00 01
04 0b 00 1e 09 e2 10 00 13 88 12 8e 00 01 14 1e
00 01 20 00 00 01 30 03 02 58 0a c8 07 a3 13 92
12 8e 40 00 07 d0 03 e8 50 08 00 01 13 9c 01 90
00 10 01 f6 13 74 70 02 00 01 27 10 80 00 00 00
08 5b 01 2c 08 b7 09 41 09 9d 01 2c 90 00 00 00
ff a1 b0 00 00 00 01 f4 00 5f a0 02 00 00 00 00 
9c 27

@lumapu @tbnobody vielleicht wollt Ihr das irgendwann demnächst mal ins Projekt einbauen ?

stefan123t commented 6 months ago

Hier ist die Prüfung der DevControl Command Antwort vom Inverter durch die DTU Pro aus dem Hoymiles Source Code Evtl. ist dies auch für andere DevControl Commands wie Power Limit, etc. interessant ?

RX 9.041319200000000,0x7E 8A 80722536 74438179 81 000003002000 9F19 80 7F
                                                  ^^ --------------------- pBuffer[11] Uart_CurrentReplyState
                                                      ^^^^ --------------- pBuffer[13][14] Uart_CurSendSubCmd

/***********************************************
** Function name: Three-generation protocol device control application layer receipt processing
** Descriptions:
** input parameters:    ?
** output parameters:   ?
** Returned value:      ?
*************************************************/
void UsartNrf3_Process_DevControl(u8 *pBuffer)
{
    Uart_CurrentReplyState = pBuffer[11];

    if(Uart_CurSendSubCmd != (((u16)pBuffer[14] << 8) | pBuffer[13]))
    {
        return;
    }

    switch(Uart_CurrentReplyState)
    {
        case 0: // The command was executed successfully

            //dong 2020-06-15 Grid-connected protection file self-check command is processed separately
            if(Uart_CurSendSubCmd ==  Type_SelfInspection) // Type_SelfInspection 0x28
            {
                //Improper response to self-test status, do network command processing
                MIReal[PortNO].Data.NetCmd = NET_SELF_STAUS;
                MIReal[PortNO].Data.NetStatus = NET_NOCMD;
                SelfCheckStateFlg = 1;
            }
            else
            {
                MIReal[PortNO].Data.NetStatus = NET_EXECUTION_COMPLETED;
                ProtocalLayer_Cmd = InitLayer;
                Index_Para = 0;
                CurNetCmd = NET_INIT;
            }

            break;

        case 3:  //eeprom read and write error
            Index_Para = 0;
            break;

        case 4:  //This command is not supported
            MIReal[PortNO].Data.NetStatus = NET_EXECUTION_FAILURE;
            ProtocalLayer_Cmd = InitLayer;
            Index_Para = 0;
            CurNetCmd = NET_INIT;
            break;

        case 7:  // Abnormal length of configuration parameter
            Index_Para = 0;
            break;

        default:
            break;
    }
}

stefan123t commented 3 weeks ago

https://github.com/lumapu/ahoy/issues/365#issuecomment-1899405729

Prima! Danke an nivadis für den Dump beim Grid Profile DOWN_DAT 0x0A Kommando: ... @lumapu @tbnobody vielleicht wollt Ihr das irgendwann demnächst mal ins Projekt einbauen ?

I have double checked it, this is the upload of a DE_VDE4105_2018 grid profile as documented already elsewhere in our S-Miles_Cloud_Grid_Profles.xlsx Sheet:

03 00 20 00 00 0A 08 FC 07 30 00 1E 0B 3B 00 01 
04 0B 00 1E 09 E2 10 00 13 88 12 8E 00 01 14 1E 
00 01 20 00 00 01 30 03 02 58 0A C8 07 A3 13 92 
12 8E 40 00 07 D0 03 E8 50 08 00 01 13 9C 01 90 
00 10 01 F6 13 74 70 02 00 01 27 10 80 00 00 00 
08 5B 01 2C 08 B7 09 41 09 9D 01 2C 90 00 00 00 
FF A1 B0 00 00 00 01 F4 00 5F A0 02 00 00 00 00

So it is the usual multi command series but this time from DTU to Inverter starting with 0x0A 0x01 .. 0x0A 0x88:

0A <inv> <dtu> 01 <grid profile in 16 byte chunks> <crc8>
0A <inv> <dtu> 02 <grid profile in 16 byte chunks> <crc8>
...
0A <inv> <dtu> 07 <grid profile in 16 byte chunks> <crc8>
0A <inv> <dtu> 88 <crc16 of grid profile> <crc8>

Then the DTU repeats the last packet 0x08 of the DOWN_DAT 0x0A command with the <crc16 of grid profile> repeated every 100ms for some time ~2,4s alone without response. ΔT 2.4009187120602133 s

0A <inv> <dtu> 08 <crc16 of grid profile> <crc8>

Then the Inverter will chime in and reply 0x8A 0x81 with the grid profile signature and a crc16 of the 00 00 <grid profile signature> reply for some more time ~10s. ΔT 9.7781032 s

0A <inv> <dtu> 88 <crc16 of grid profile> <crc8>
8A <inv> <dtu> 81 00 00 <grid profile signature> <crc16> 80

After that the DTU will continue to send the above crc16 of grid profile though the Inverter will stay again silent for some 2,46s. ΔT 2.4652518400000005 s

0A <inv> <dtu> 08 <crc16 of grid profile> <crc8>

The Inverter will start to respond again presumably after reboot (2.46s) to the previous tune of 0x0A 0x88 and his 0x8A 0x81 response for another 5.29s. ΔT 5.295623441090427 s

0A <inv> <dtu> 88 <crc16 of grid profile> <crc8>
8A <inv> <dtu> 81 00 00 <grid profile signature> <crc16> 80

So in total this final echo command 0x0A 0x88 of the DOWN_DAT and the Inverters reply 0x8A 0x81 will be repeated every 100ms until the Inverter has flashed the new Grid Profile at the right location, which takes about 20 seconds in total. ΔT 20.308987587 s

Here you can see the two gaps in the repsonse from the Inverter in channel 14 vs. the DTU in channel 6.

There is a short gap of 120-130ms after these 20,3s and then it is following in the common 0x15 0x0B RealTimeRunData_Debug command request / reply cycle as usual.

15 <inv> <dtu> 80 0B00 65A9 393B 0000 0000 0000 0000 <crc16> <crc8>
95 <inv> <dtu> 01 0001 00F9 012C 02E9 0000 01A0 000A 39F6 <crc8>
95 <inv> <dtu> 82 1386 02C2 0001 001D 03E8 0149 0002 <crc16> <crc8>
...

BTW here are the timings of some of the requests as recorded by the Salea Logic 2 shrimp. So these signal timings in seconds should be pretty precise.

26.676983679999999 request 26.724774239999999 reply 01 26.773686560000002 reply 82

27.127994399999999 request (next) 27.174777760000001 reply 01 27.223690240000000 reply 02

There is a gap of 0,04779056 seconds or 47 ms between the start of the command and the reply being received. Actually 41,95132 ms between the end of the command and the start of the reply, and another 46,65648 ms between end of reply 01 and start of reply 82. After end of reply 82 it takes 353,95648 ms before the next command is sent.

stefan123t commented 3 weeks ago

Flash known Grid Profiles

@tbnobody as a first step we could allow to select / flash the most common grid profiles which are already known.

I have documented the known grid profiles along with the valid ranges as allowed by the Hoymiles Cloud UI here: https://github.com/lumapu/ahoy/wiki/Protocol#file-name-at_tor_erzeuger_default

Modify Grid Profile within known/allowed Ranges

In a second step we would need to verify and implement the ranges for the different grid profile versions. And make the values which are allowed to be changed adjustable within those ranges. I have marked adjustable values shown as digits, e.g. 255.3 (signalling an input box) with a given range other than ~.

For some of the Grid Profiles there are boolean variables showing 0~1 as range. https://github.com/lumapu/ahoy/wiki/Protocol#file-name-de_vde4105_2018

Also in the Details DE_VDE4105_2018 there is Reactive Power (VAR) under Reactive Power Control (RPC) which can be leading / lagging. This is shown by the caret ^ / v characters given in my documentation. This could probably be better explained using a textual tool tip or a boolean leading (* ) / lagging ( *) switch in the UI.

abuesen commented 1 week ago

Brauche diese Funktion auch.

Seit gestern habe ich einen HMT-2250 zusätzlich in Betrieb. Profil ist EN 50549-1:2019. Hab dadurch erst auf die Spannung geschaut und tatsächlich zeigt die Messung am EHZ bereits Werte von 250V+, selbst wenn ich beide PV-Systeme trenne. Die historischen Werte zeigen regelmäßig 250V an sonnigen Tagen.

Die Hauptanlage hat noch nie abgeregelt, ein FoxESS mit Profil VDE-AR-N 4105.

stefan123t commented 1 week ago

@abuesen Du kannst mit den Original Hoymiles DTUs das Grid Profile ändern oder sogar innerhalb bestimmter Grenzen anpassen. Das habe ich oben prinzipiell unter 1. und 2. beschrieben wie man es für die OpenDTU/Ahoy ebenfalls implementieren könnte.

Die EN 50549-1:2019 ist prinzipiell in der EU zugelassen da es sich um eine Vereinheitlichung der nationalen europäischen Standards handelt. Dass der WR mit dem neueren EN Grid Profile ggü dem ein Jahr älteren, nationalen deutschen DE VDE4105:2018 etwas früher abregelt ist schade aber mE aufgrund Vereinheitlichung europäischer Normen verständlich und hinnehmbar. Sonst musst Du eben das og Feature implementieren, Dich bei Hoymiles / Deinem Händler beschweren oder eine Original Hoymiles DTU kaufen. 🤷

https://www.researchgate.net/profile/Roland-Bruendlinger/publication/338800967_Grid_Codes_in_Europe_-_Overview_on_the_current_requirements_in_European_codes_and_national_interconnection_standards/links/5e2aeea24585150ee77f6be9/Grid-Codes-in-Europe-Overview-on-the-current-requirements-in-European-codes-and-national-interconnection-standards.pdf

tbnobody / OpenDTU