UtilitechAS / amsreader-firmware

ESP8266 and ESP32 compatible firmware to read, interpret and publish data to MQTT from smart electrical meters, both DLMS and DSMR is supported
Other
379 stars 72 forks source link

Kamstrup whit current transformer #216

Closed oz1iio closed 2 years ago

oz1iio commented 2 years ago

Is your feature request related to a problem? Please describe. I use your brilliant solution at home, works perfekt whit my Kamstrup meter + HomeAssistent, but, at my work at Nordic Folkecenter for Renewable Energy, we got 250A fuses and approx total 100kw PV and windturbine, demonstattion and testing.

Describe the solution you'd like The limit Main fuse at 63A, its possible to change, so I can set it higher, same for 50Kw feed in.

Additional context When I set fuse for 63A the grafical readOut at AMS reader seams correct, yes current read, but the value readout is wrong. Not running whit MQTT yet, maybe the data is OK this way. I use a DOIT ECP32 board.

My plane is to promote your software and help people to upload the software and use it together whit small Windturbine and PV installation, for controlling heat and EV charging. Hope that's OK.

gskjold commented 2 years ago

Sounds like a interesting setup you have there :) 'll just change both fields to open range between 0 and 255. You should get correct readings regardless of fuse size and production capacity. Try the attached firmware and set correct fuse and production. Let me know if everything looks ok after. If not, send me a screenshot.

Where are you located?

Feel free to help people set up and use this software for personal applications. Beyond this scope (commercial use) I would appreciate setting up an agreement.

esp32.zip esp8266.zip

Shadrack88 commented 2 years ago

is it possible to use the firmware for kampstrump meter that has a fuse with more than 3000A

oz1iio commented 2 years ago

Thanks for quick reply and update of software. It seems the current is readout in a facktor 10. I have no idea why.

I am locate in DK, the north west part of Jutland. The DSO is ThyMors energy and I have no problem to get the GPK codes and aktivet HAN push.

AMS_v2 0 7_current

ArnieO commented 2 years ago

@gskjold I believe the issue here is that installations like the ones @oz1iio and @Shadrack88 mention are commercial/industrial sites with a different set of current transformers installed (or else the input on the meter would be saturated). Then I assume the electricity company knows they have to multiply the readings by (for instance) a factor 10x for to get the actual consumption.

Opening the "Main fuse" selection to any value up to 255 is a good point that generalizes the firmware - but will not solve that issue.

I would advise against opening up for "tampering" with the readout from the meter, it is a potential "slippery slope".

gskjold commented 2 years ago

Aha, I get it now, thanks. There is probably something in the payload to suggest what scaling is used.

@oz1iio any chance you could enable debugger to verbose level and post me the output? Log can be accessed from either telnet or serial (usb)

gskjold commented 2 years ago

@Shadrack88 technically, yes, but the graphs would be all weird because the configuration can only scale to 255A fuse and 255kWp production. It doesn't cost me much time to expand the range though if you want to test? Also you will probably have the same scaling issues mentioned above, hopefully there is something in the payload that reveals what scaling is used.

ArnieO commented 2 years ago

There is probably something in the payload to suggest what scaling is used.

It would indeed be awesome if there is a scaling factor in the payload.

ArnieO commented 2 years ago

@Shadrack88 : Can you too provide a Telnet debug from your high-current meter, as indicated in https://github.com/gskjold/AmsToMqttBridge/issues/216#issuecomment-1026564449?

Shadrack88 commented 2 years ago

is it possible to scale the the max current of main fuse beyond 63A after installing the new firmware update?

Shadrack88 commented 2 years ago

@ArnieO how can i provide a telenet debug from my high current meter?

oz1iio commented 2 years ago

(D) (AmsWebServer)Serving /data.json over http... (D) (AmsWebServer)Serving /data.json over http... (D) (readHanPort)(C1) Frame dump (461b): (D) 7E A1 CB 2B 21 13 57 E7 E6 E7 00 0F 00 00 00 00 (D) 0C 07 E6 02 01 02 09 11 14 FF 80 00 00 02 41 0A (D) 0E 4B 61 6D 73 74 72 75 70 5F 56 30 30 30 31 09 (D) 06 01 01 01 08 00 FF 06 00 C2 D1 70 09 06 01 01 (D) 02 08 00 FF 06 01 3B D2 95 09 06 01 01 03 08 00 (D) FF 06 08 9D 28 BE 09 06 01 01 04 08 00 FF 06 00 (D) 01 35 1D 09 06 01 01 00 00 01 FF 06 01 97 EC 48 (D) 09 06 01 01 01 07 00 FF 06 00 00 00 00 09 06 01 (D) 01 02 07 00 FF 06 00 00 0A 19 09 06 01 01 03 07 (D) 00 FF 06 00 00 F7 84 09 06 01 01 04 07 00 FF 06 (D) 00 00 00 00 09 06 00 01 01 00 00 FF 09 0C 07 E6 (D) 02 01 02 09 11 14 FF 80 00 00 09 06 01 01 20 07 (D) 00 FF 12 00 E9 09 06 01 01 34 07 00 FF 12 00 E9 (D) 09 06 01 01 48 07 00 FF 12 00 E9 09 06 01 01 1F (D) 07 00 FF 06 00 01 63 77 09 06 01 01 33 07 00 FF (D) 06 00 01 6F 03 09 06 01 01 47 07 00 FF 06 00 01 (D) 69 E5 09 06 01 01 15 07 00 FF 06 00 00 00 00 09 (D) 06 01 01 29 07 00 FF 06 00 00 00 00 09 06 01 01 (D) 3D 07 00 FF 06 00 00 00 00 09 06 01 01 21 07 00 (D) FF 12 00 02 09 06 01 01 35 07 00 FF 12 00 05 09 (D) 06 01 01 49 07 00 FF 12 00 04 09 06 01 01 0D 07 (D) 00 FF 12 00 04 09 06 01 01 16 07 00 FF 06 00 00 (D) 01 C0 09 06 01 01 2A 07 00 FF 06 00 00 04 6D 09 (D) 06 01 01 3E 07 00 FF 06 00 00 03 EC 09 06 01 01 (D) 16 08 00 FF 06 00 6F 8E 9E 09 06 01 01 2A 08 00 (D) FF 06 00 6B A1 B6 09 06 01 01 3E 08 00 FF 06 00 (D) 76 8B B5 09 06 01 01 15 08 00 FF 06 00 34 58 05 (D) 09 06 01 01 29 08 00 FF 06 00 41 F1 F4 09 06 01 (D) 01 3D 08 00 FF 06 00 62 70 EC D0 2A 7E (D) (readHanPort)(C1) Valid data, start at byte 29 (V) (AmsDataStorage) Time is: 1643703443

image

gskjold commented 2 years ago

Thanks, a typ-o in the markup prevents update of export value.

Unfortunately there is no scaling information in the data received. I guess they just change the current transformer and leave the data as it is..

tronde-ams commented 2 years ago

@oz1iio Do you have a type number for the Kamstrup meter?

oz1iio commented 2 years ago

Your absolute right. It's standing at the front.

image

oz1iio commented 2 years ago

We got a good wind this night and I can see at the production from the turbine, it's had increased the production and drop again around 7 am Looks like the day graph has a overflow at 32kwh/hours But not sure.

image

ArnieO commented 2 years ago

Looks like the day graph has a overflow at 32kwh/hours

@gskjold Could this be an overflow error when exceeding negative 32768 Wh (signed integer data type)?

gskjold commented 2 years ago

correct, int16 overflow

tronde-ams commented 2 years ago

I found the datasheet for Omnipower CT.

omnipower ct (v. 2021) 58101313_H1_GB.pdf

It says the meter can be programmed with a scaling factor

trafo 2022-02-02_173816

According to this table, and the 5A marking on the meters name plate and use of 250A fuses, I guess the scaling factor is 60 or maybe 100

ratio 2022-02-02_173740

I have searched for information about the Danish HAN-port, but found almost nothing, except this from retsinformation.dk https://www.retsinformation.dk/eli/lta/2019/75

§ 5. En fjernaflæst elmåler skal være i stand til at videreformidle målte data om aftag og levering af elektricitet for de enkelte tidsserier til netvirksomheden og en ekstern enhed. Den fjernaflæste elmåler skal som minimum kunne bringes til at vise akkumulerede værdier for aftag og levering af elektricitet samt den aktuelle elektriske effekt.


§ 7. Det skal være muligt for forbrugeren efter åbne standarder at tilkoble eksterne enheder til den fjernaflæste elmåler og løbende udtage forbrugsrelevante data, jf. § 5.

Stk. 2. Overførsel af data skal være sikret via kryptering eller lignende.

Stk. 3. Tilkobling af eksterne enheder skal kunne foretages af lægmand uden at bryde plomber eller lignende. Netvirksomheden kan beslutte, at aktivering skal foretages eksternt af netvirksomheden. Dette må ikke påføre forbrugeren unødige ekstraomkostninger.

Stk. 4. Udtagning af data må ikke være til gene for netvirksomheders brug af den fjernaflæste elmåler.

Since it seems to be a legal requirement to make at least some values from the meter available to the customer (en ekstern enhed), I suggest to ask the grid company about the details before any changes to the code are made. .

oz1iio commented 2 years ago

It's Kamstrup we have to go for The DSO grid operator knows very little about the meter

Normal you can find all information directly at Kamstrup's homepage.

If we know precisely what we missing, I can contact Kamstrup.

tronde-ams commented 2 years ago

I have spent some time at their homepage, but not found anything useful about the data available at the HAN-port. It can still be hidden there.

In the user manual, it is described how to display the current transformer's ratio / scaling factor. _Unfortunately it seems like this meter has a choice between 12 different display confugurations, so I can't tell exactly how to here.

You find all 12 descriptions under "Brugervejledning (24)" https://www.kamstrup.com/da-dk/elloesninger/intelligente-elmaalere/meters/04-omnipower-ct/documents

I suggest you make a table showing what data you have (what values you actually get from this adapter) and what you expected it to be. Then it is much easier to get a picture of how things are related and much easier than to connect a lot of text.

I understand that your voltages appears to be OK. This makes sense since they will never be subject to any scaling because of a current transformer.

Type of data ..................................................From adapter..........................Expected value.

Instant imported power Instant exported power

Accumulated imported power Accumulated exported power

Current

tronde-ams commented 2 years ago

I came across another document stating this about OBIS and current transformers if it makes some sense

obis

Page 72

omnipower technical description.pdf

gskjold commented 2 years ago

Great find, that means there are potential hope for CT meters. Unfortunately it is not present in the frame dump from this meter.. Could it be a configuration error from the grid company?

tronde-ams commented 2 years ago

It can also be that they use OBIS codes to program the functionality of the meter. The table shown above seems to be for display configuration.

But - do we need transformer ratio as OBIS? We know there is a ratio in use, If this ratio can be found, it should be possible to configure the module so it will use that ratio?

If we are lucky, the documentation is correct about how to display the ratio in the display as I wrote two postings before this. @oz1iio must check and tell us.

It is also most likely that other manufacturers will use the same ratios as Kamstrup (shown in the light blue table above).

Omnipower CT is delivered in two versions. One for 1A input and one (this particular meter) for 5A input. Since long time ago analog amp-meters used with current transformers are most likely also either 1A or 5A types, so I guess they use standard transformers. From Kamstrup's technical description (p 11) it seems like the Norwegian Aidon 6550 is a 1A meter.

ArnieO commented 2 years ago

My opinion is (or has at least been) that the module shall deliver data coming from the meter, without any "tampering". If there is an OBIS code in the payload that defines the scaling of that meter: Fine, we multiply. But I do not think it is a good idea to add a user adjustable scaling factor parameter in the GUI.

tronde-ams commented 2 years ago

I don't see a configurable scaling factor as a big problem if it easy to implement.

This module cannot legally be used for anything fiscal so why not make it useable for more people? In Norway most consumers will probably not have a CT-meter, but I know there are a lot of privately owned small wind turbines in Denmark which probably will requre a CT-meter.

ArnieO commented 2 years ago

Good point, thank you. Adding support for CT meters will also make the firmware useful for meters in other non-residential installations (industrial, commercial, ...). So will definitively render the firmware more versatile. Let us hear what @gskjold thinks regarding implementation. If added as an option I think it must be a separate "section" that opens after user has confirmed "Current transformer" (similar to the current implementation of Static IP). The aim is to minimize the risk of accidentally changing the scaling parameter.

tronde-ams commented 2 years ago

Since @oz1iio is here with a real system and a CT-meter, I think it can be useful to investigate what is possible.

I am more than willing to write the help section for how to configure the adapter for use with a CT-meter if the test gives useful results.

oz1iio commented 2 years ago

Sorry for my delay

  1. Volt is correct as we know

  2. Watt is correct too, we have scada systems on our wind turbine, and it produced approx 60kw and 120A on the phase's image We have a electrical boiler for heat, 37kw at full load, else the boiler balances the power. So it's not a factor wrong, seems correct.

  3. The current readout is a factor 10 wrong, and it's have nothing with the CT to do. image Our grid operator DSO can set the ct factor remote, and it's set to 60. And our reading is definitely not a factor 60 wrong The factor is set in the kamstrup meter, and stored in the meter.

Hope it can help.

tronde-ams commented 2 years ago

Is it realistic that the current is a factor 10 off? Wrong interpretation of decimals?

This makes sense from your first screenshot with 6.4 kW import and about 9A average current / 235V.

It can be that these low-current meters don't present values for current the same way as the meters for direct connection.

gskjold commented 2 years ago

I have restored some current divider calculation that i had in v2.0.0, please try the version below:

esp32.zip esp8266.zip

oz1iio commented 2 years ago

image

Looks like it's okay, but it's not

The turbine and the boiler is in power balance, but current is still around image

So now I think the current reading is too low. Special if you looking at the Var value on the first picture . A factor 100 to low ?

gskjold commented 2 years ago

Alright, I see what you mean. I made another adjustment. Not really ideal code, but could still work.

esp32.zip esp8266.zip

oz1iio commented 2 years ago

image image image image

I have no clue. I know the wind turbine keep the current between 90-120A approx

oz1iio commented 2 years ago

The strange thing is that the current valve make jumps. Only system I can see, if current below approx 98A 9,8A 0,98A it's made a jump, But it's not a fix. Can be 1A and jump to 10.1A and again up or down. So maybe it's not a scale factor but something else.

tronde-ams commented 2 years ago

@gskjold What is the current divider calculations you mentioned?

I can't exactly see any need for calculations with a meter for a three phase TN-system. The very first screen shot with 6.4 kW import indicates the only error is the currents are 10x off (too large) when we also consider the 527 VAr import.

I think we must be absolutely sure about the resolution of the current's values presented on the HAN. Are they the same for this meter as they are for the directly connected meters? Because of the very low initial current range, it makes sense if this CT-meter has 10x resolution compared to the other meters.

tronde-ams commented 2 years ago

There is something that differs between direct coupled and transformer coupled Omnipower meters. This is from page 9 https://www.nek.no/wp-content/uploads/2021/11/5512-2441_GB_C1_06-2020_JAS_Original-version-5ee71c9e9d938.pdf

It does not say much to me, but it has to do with currents and data pushed to HAN. It is the same in the table on p. 10.

ct scaler
gskjold commented 2 years ago

Current divider calc happens here: https://github.com/gskjold/AmsToMqttBridge/blob/db5e242ad8cfb6b16775baf22c063e0627529958/src/IEC6205675.cpp#L312-L321

Your last screenshot makes sense for this meter, but without any identifier in the payload there is no way for the software to detect that this is a CT meter. Meter model is not included in payload

@oz1iio You mentioned GPK keys, but from the payload you posted earlier it looks like data from your meter is not encrypted, can you confirm?

oz1iio commented 2 years ago

It's encrypted exactly as my own at home. Same setup

gskjold commented 2 years ago

Have you tried removing encryption key from the setup and see if it still works? I don't see how my software could have detected that this is encrypted and used the encryption keys at all.

oz1iio commented 2 years ago

Just remove the key and it's still working.

tronde-ams commented 2 years ago

Is it only in Denmark they don't push any identifier in the payload? How does the code distinguish between single phase and three phase meters, or the Norwegian IT-meters form TN?

I flashed a new ESP now, and it seems like you rely entirely on data in the payload for identification? If so, it should be possible to add a manual choice of "Kamstrup Omnipower CT meter". "CT meter" is only for current transformers.

It is also possible to read this from both the type and type number printed on the name plate.

kamstrup type

My understanding of this specification (scale 2 / 3), is that the payload for current has a fixed number of digits (5?), and that the two last are decimals if a direct connected meter, and that the three last are decimals if a CT-meter.

This imples that a directly connected meter will push current without any need of scaling (with 0,01 A resolution) while a CT-meter must use x10 scaling to show the real value for the user.

I am also a little puzzled you calculate watt. Does it exist any meter not pushing watt in the payload?

gskjold commented 2 years ago

Is it only in Denmark they don't push any identifier in the payload?

As far as I know.

How does the code distinguish between single phase and three phase meters, or the Norwegian IT-meters form TN?

It looks for OBIS codes for voltage and current, if all three are present, it is a 3p meter. It cannot distinguish IT from TN, hence the setting in meter config.

I am also a little puzzled you calculate watt. Does it exist any meter not pushing watt in the payload?

No, all meters publish watt. The calculated value is used to check if current has a sane value, if not divide by 10 until it is sane. Don't worry, I wont keep this code.

tronde-ams commented 2 years ago

I see. Sanity checks makes good sense.

I think I understand what they try to tell with this "scaler" value shown in the tables in the NVE-document.

0 means multiply by 10E0 = 1 1 means 10E1 = 10 -2 means 10E-2 = 0.01 -3 means 10E-3 = 0.001

I am quite certain Kamstrup CT-meters are off by 10x for currents, and that it can be possible to "find" them by comparing calculated current values with the values pushed to HAN. One problem by doing so, is that these meters will most likely be found in installations with both high import and export values, and also quite high values of reactive power which can give unexpected values, but not as much as 10x off. If you allow for some variations, I guess it will work out OK.

tronde-ams commented 2 years ago

They don't happen to have meter type as part of the serial number?

684 685 686

or x84 x85 x86

oz1iio commented 2 years ago

image Can see there is a drop down on HAN push liste. No idea, and not sure if they will change anything, I just have a screen shot. Give no sense for me, just if you can see anything.

ArnieO commented 2 years ago

Just remove the key and it's still working.

@oz1iio --> ...?

Also @oz1iio : Where are those screendumps from?

oz1iio commented 2 years ago

From the DSO operator, where they set up the meter. They send me the screenshot to show me the HAN bus is Active. I have no access for this.

Yes I have notes there is no X in encryption

ArnieO commented 2 years ago

From the DSO operator, where they set up the meter.

OK, understood.

Yes I have notes there is no X in encryption

I probably misunderstood you, but got the impression that you have an encryption key but your meter is not encrypted / the reader works without it?

oz1iio commented 2 years ago

Correct, I get a key and use it, but gskjold asked me to delete the key, and it still work.

gskjold commented 2 years ago

Interesting screenshot, I wonder what they have in "HAN-push-liste". You should recruit this guy to start using this software so that we get an insider ;)