CircuitSetup / Expandable-6-Channel-ESP32-Energy-Meter

Hardware & Software documentation for the CircuitSetup Expandable 6 Channel ESP32 Energy Meter. Works with ESPHome and Home Assistant.
https://circuitsetup.us/product/expandable-6-channel-esp32-energy-meter/
MIT License
510 stars 101 forks source link

Different voltage readings for connected voltage inputs #88

Open mmallozzi opened 2 years ago

mmallozzi commented 2 years ago

I'm in the US with split phase 120V/240V AC power. I just set up my first expandable meter, starting with one of my subpanels where all I want to measure right now are a few 240V HVAC appliances which seem to be balanced (all newly wired with 2 wires plus ground, so no neutrals), so I figured I'd be okay with one AC transformer. So I have not cut the jumpers to enable separate transformers (I'll end up doing that for the meter on my main panel). If I understand correctly, that means they are hardwired together, and I should get the same exact reading for V1 and V2. However, I am seeing ~117V for V1 and ~127V for V2. The voltage calibration values in EmonESP are the same for both.

CircuitSetup commented 2 years ago

It's very odd to see that much of a difference between the 2 voltage readings when they are coming from the same AC transformer. I would recommend calibrating the voltages separately, and changing the calibration values so they both read about the same. Otherwise the power reading is going to be a bit off on one side of the meter.

mmallozzi commented 2 years ago

Makes sense - thanks!

Cougar commented 2 years ago

I'd like to second on this. It is not just a calibration issue. Maybe even another issue should be created.

I did a simple test with three meter boards, all connected to similar AC power supplies (could be a little bit different of course). I changed calibration values until I got very close readings from all meters. Values (for ESPHome config) were following:

gain_voltage: "12770"
gain_voltage: "12674"

gain_voltage: "12638"
gain_voltage: "12727"

gain_voltage: "12697"
gain_voltage: "12750"

Then I did power restart for all modules (moved them to another extension cord) and the voltage difference between the lowest and highest reading changed from less than 0.5V to around 5V! M90E32AS Vrms voltage accuracy by spec is ±0.5%

imgur-2022_02_23-154013

This means that it was just a pure luck that these calibration values gave me the same voltage readings. Now I'm troubleshooting this issue with all values 12770 and the result is the same. Every board reset makes readings different. Here is one ESPHome reset:

imgur-2022_03_05-15:44:57

For this graph I measure the voltage each second.

Cougar commented 2 years ago

I first suspected some sort of initialization problem because not all setup registers are written in the boot. However, based on spec it should not be a problem and defualt values are going to be used in this case. I have done many chip restarts and read all registers after that but haven't seen any difference in config registers.

Another idea came from M90E32AS datasheet 6.4 POWER ON RESET TIMING where it says:

To make sure M90E32AS is reset and can work properly, MCU must force M90E32AS into idle mode firstly and then into normal mode. In this operation, RESET is held to high in idle mode and de-asserted by delay T1 after idle-normal transition

This meter connects PM0 and PM1 to VDD which means that the chip is hardwired to Normal mode only. Could this cause a problems here? Now way to test without removing two chip pins from PCB..

CircuitSetup commented 2 years ago

@Cougar that is very interesting. I haven't observed that behavior myself. You may be onto something with the reset timing. Are you doing a reset via software or power cycling? If software, I'm wondering if it is not updating the calibration values properly.

You may want to try a reset via software, and compare to power cycling to see if there is a difference. I'm going to bet there is. Especially with the second IC (assuming there is a natural bit of delay in initializing the second IC) You are correct that the board puts the metering ICs in normal mode only, but I haven't seen problems with this in the past. It may be a matter of reinitializing and writing to the config register with a POR. The code for that in ESPHome is located here.

Cougar commented 2 years ago

@CircuitSetup My first post was power cycle. I moved the whole setup to another room and lost all my "good" calibrations.

Most resets are software resets that ESPHome does when uploading new firmware. I hope to find something and add more and more debug to this firmware. Now I dump all registers to the log after chip initialization. Only differences are actual measurement registers, all config remains unchanged.

Now I did 10 sec power cycle for all three meters and this is how it looks like:

emon-reset

"Line Voltage" is the UrmsA of the first IC and "Line2 Voltage" is second IC.

I also tried to set up all offset registers based on this code but it didn't change anything.

CircuitSetup commented 2 years ago

I would expect that the 3 voltage measurements per IC would be similar to each other since they have the same voltage dividers (there's a set of voltage dividers for each IC). Depending on when you received your meter those resistors are either 1% or 0.5%. More recent batches are 0.5%. I'm guessing the variance between individual IC voltage readings is either the IC itself, or the small delay in ESPHome actually taking the reading.

Either way, if you're reading power from the meter, and each voltage is calibrated, you should still get very accurate results.

FYI, I believe the offset registers only apply to the accumulated power registers. The datasheet is a bit confusing regarding them, and how they should be used.

Cougar commented 2 years ago

I 100% agree. I read only first voltage from every chip. I will try to read all three to see if there is any inconsistency between them too. Of course reading times are slightly different but from graph it is still visible if the difference has always the same offset.

I don't think that voltage divider accuracy is a problem right now because the change always happens with chip reset. I'll add some more knobs to ESPHome and check if also happens if I just send ATM90E32_REGISTER_SOFTRESET without any other configuration in setup phase. Or if ATM90E32_REGISTER_METEREN toggle change anything.

If I understand correctly then all config goes to flash and you don't need to resend them at all. This should be the reason for config CRC register and dedicated WarnOut pin. Isn't it how simple pulse counting meters with this chip (and no MCU) should work?

Cougar commented 2 years ago

I tried to reset chips during operation and this didn't change anything. However, the whole ESPHome reset almost always did.

My transformer is 9.0V 0.67A (6.0W) which should be enough for ESP32. But the secondary resistance 2.5 ohm is too big for a stable output for ESP32 current peaks which are by datasheet up to 240 mA for wifi itself. This is at least 0.6V drop already. Most probably the difference between two IC readings came from timing when ESP32 wifi is transmitting or not.

When I removed the ESP32 from PCB and powered it from USB (keep only M90E32AS power from AC), the difference between all voltage readigs is around 0.2-0.4V only.

emon3_voltage

Now I'm thinking, should I remove rectifier from PCB, keep AC only for voltage sensing and use separate USB power adapter for power the boards to get even more precise reading.

Would be great to have dedicated jumper traces for such configuration in the future PCB revision. Or even better, change the JP12 and JP13 to three position so that it is possible cut J4 AC socket from VA+ and VA- and use J3 for both ICs.

imgur-2022_03_12-21:13:54

alexruffell commented 1 year ago

I am not sure whether it is the same issue or not, but it looks like it may be.

I have a tiny 9V PCB mount transformer to provide reference for V2, and a PCB mounted 12V 14VA transformer to power the board and ESP32, and provide V1. While I do see a voltage drop of about 0.1V when I connect the ESP/energy monitor to the 12VAC transformer, it tends to be consistent (meaning it seems -0.1V all the time but hard to tell for sure given the fast line voltage fluctuations). I am using a 5 1/2 digit bench DMM set to slow for my measurements. I compare the DMM measurement of line voltage (which for now is powering both transformers) to the voltage reported back for V1 and V2 and calibrated them to show the same reading. Every time I go back to it after some time, the delta between V1 and V2 is off by up to 0.5V while after calibration it was within 0.0V and 0.1V. I chart the voltage readings in Grafana and it is quite visible that they grow apart.

I often noticed that when I reset the board manually or by uploading a new ESPHome firmware, the calibration change I made seemed off, or the calibration I had completed was now off again. This sounds similar to what @Cougar was reporting although with a smaller delta.

I know that in the big scheme of things a voltage measurement error up to 0.5V on a 120V line is not critical but I was wondering whether anything could be done. I just purchased my board and have v1.4 rev1. Are there any changes I can make to it to make things better (better resistors, etc)? I am comfortable with soldering and have good enough equipment to do it.

Also, is there something I can do to stabilize the secondary output powering the ESP so that V1 won't be affected by its fluctuating power draw?

alexruffell commented 1 year ago

Based on the graph below, the delta may have something to do with the update delay. When the delta is set to the normal 10s delay, the delta between V1 and V2 is 0.4 ~ 0.5V but when it is set to 2s it is 0.1V. So, the calibration should be fine in my case, just a matter of timing and the measurements not being taken close enough in time. I wonder whether there is some way to improve on that without keeping them at 2s which bogs down my HA with tons of db writes and unnecessary data.

In the graph below, where the two lines grow nearer is when I uploaded the firmware changed from 10s to 2s. The big L1 dips were when I rebooted the ESP remotely which may be connected with a higher draw on the transformer while ESP32 is booting up causing a minor drop that translates to a bigger drop of this graph.

image
Cougar commented 1 year ago

I modified my board a little bit to remove board power from AC transformer. The easiest was to just remove a rectifier from the board and supply power to ESP and ATM90E32AS via NodeMCU micro-USB port. I bought dedicated DIN rail 5V power supply for that. I'm not sure if this is a good practice to feed 3.3V to the step down switcher but so far it works fine. Might be better to remove L1 as well to completely disconnect onboard power supply but this component is much harder to remove.

alexruffell commented 1 year ago

@Cougar - Thanks for the tip. For now I have a large capacitor on the 3.3V and 5V but I still have to vet if it is really helping. I opened a ticket on ESPHome github reporting a related issue. I have 2 boards... and the 1st chip of both boards alternate having a 0.5V error in the reading.

image

The arrows indicate manual restarts, no changes. For an instance they all matched but that rarely happens. I can keep restarting and the L1 V and L1 AO V just keep trading places having the 0.5V error. That is not due to the ESP32 current draw, or voltage fluctuation etc... it has to be something with how the chips are configured (?) at boot-up.

Cougar commented 1 year ago

My conclusion was that it doesn't matter if the problem is visible for one or both chips. These chips are polled in different times and it is possible that during one operation the ESP32 draws more current than during another. I think it is because of WiFi but I haven't tried to measure these timings to prove it. WiFi just seems most probable reason due quite high current draw.

alexruffell commented 1 year ago

@Cougar What I am showing above is not due to wifi as only 1 of the 2 voltage channels (chip1 on both boards) connected to the same transformer see the 0.5V shift and that shift doesn't change based on anything other than a reboot. The behavior is so repeatable that I am convinced there is a software bug somewhere during initial setup of the chips.

Cougar commented 1 year ago

Did you switch off the Wifi? How do you get these readings? Different boards have been read in different times. You can try to swap chips under sensor: and see if the shift moves along to another chip.

alexruffell commented 1 year ago

@Cougar No, WIFI is fully operational. For testing purposes I connected the same 120V feed to both transformers and calibrated them to match what I was reading from my benchtop 5 1/2 digit DMM. That in itself was a bit hard due to the constant fluctuations of line voltage and I found that late at night it is a bit easier. I also sped up the reporting to 3s (faster would cause it all to choke up) because I figured that the 4 chips taking measurements at different times would account for some variation. Using Grafana graphs to monitor the changes helped as I focused mostly on it trending in the right ballpark and for all 4 channels to be close to each other since they were all reading the same line voltage.

image

Trying to reword what I said in my previous post, all 4 channels are now calibrated to read the correct voltage however each time I reboot the system, ch1 of board 1 or board 2 shifts up by about 0.5V. The shift is constant and comes and goes simply by rebooting the board. Most commonly they trade places... when one is shifted high, the other is correct, and after a reboot they trade places. In one instance shown in the screenshot I re-pasted below they happened to both not have the shift so all 4 channels were reporting the voltage correctly (look at signals between the red arrows). Before the left red arrow L1V (Green) is shifted high, and after rebooting twice, L1 AO V (Blue) was now shifted high. This behavior cannot be related to what the ESP is doing... so it must be some software bug in how the chips are setup at boot (or something along those lines).

image

Cougar commented 1 year ago

Ok, now I see that I described a little bit different situation. You have quite powerful transformer and the load of the ESP and power measurement boards don't affect the reading as much as my 6W transformer (around 2 volt or 1% power drop).

I did test with all my three boards now. EMON-1 and EMON-2 are using 9V 0.67A (6W) AC power transformer. EMON-3 has rectifier removed and external transformer doesn't supply power to the board but only to the ATM90E32AS voltage sense inputs. ESP and energy meter board get their power via ESP board 5V USB input.

This is a 30 min graph with 1 second step. I reset all boards via API in 5 min interval. First graph shows voltage reading from all boards and then there are graph for each board where you can see voltage reading difference from IC1 and IC2 in volts and in percentage.

emon_voltage

You can see that the difference between EMON-1 and EMON-2 is quite big and not very stable. This is where the ESP32 power consumption changes are visible. Fluctuation is in volts.

EMON-3 does not use power from transformer and two chips should get exactly the same voltage reading. Or with small constant offset due voltage divider resistor tolerances.

Difference between two chips is very stable. Still, every reset changes the difference and I think this is the error you are seeing too and I don't have any good explanation for that either.

Cougar commented 1 year ago

I have had now some time to let my test run days.

My ESPHome is still not stable enough to run it long time. Its measuring scheduler just stops after some time and only restart helps (even over API). But this is different story and I'll take it to the ESPHome development.

Back to the ATM90E32AS errors now. I still have the issue that almost every ESPHome restart changes the error margin in the range of 2V and I have no idea what is the cause.

I made a simple button where I can run IC setup any time:

button:
  - platform: template
    name: ${node_name} IC Setup
    on_press:
      then:
        lambda: !lambda |-
          id(chip1).setup();
          id(chip2).setup();

AFAIK this button runs exactly the same IC initialization code that ESPHome reset does and still only ESPHome reset changes the voltage reading error.

But what is even more interesting is that the difference does not stay constant even between restarts and its change looks cyclic. I'm totally puzzled now.

This is how it looks like for 3.4 days continuous measurements every second where I already compensated the mean difference (0.25 V). Does anyone have any idea what is going on there?

emon

I already suspected things from ESP32 board interference and room temperature to rotation of the Earth. I already separated ESP32 board from the ATM90E32AS board with 20cm cable but nothing changed. The relative difference stays between ± 0.16 % which is very good and I actually don't worry about that but it is still interesting.

CircuitSetup commented 1 year ago

There is a feature on the energy metering IC's that compensate for readings based on temperature (which is why you can get a temp value from them). It is set to its default settings, and shouldn't need to be adjusted though. Details are here on page 35-37 Keep in mind that each chip may be reading a slightly different temperature.

I'm guessing the differences you are seeing are a combination of this and other variables mentioned above.

Cougar commented 1 year ago

Yeah but the temperature is not changing like that and it is quite stable over time. This was the first thing I checked. My daily routine is not same every day either.

Still this small change is not an issue but just interesting thing that you usually don't notice unless you do a lot of measurements.

This is how the same voltage difference graph looks like now when I send ESPHome reset in every 5 min and this is much bigger issue and can't be related with Earth rotation or anything like that :)

emon

alexruffell commented 1 year ago

@Cougar I am certain that the issue I am seeing is not caused by any other signal or environmental factor as I believe you have eliminated as well. It is so repeatable by simply resetting the board that I'd say it is certainly something to do with the software or some 'defect' in the chip. The error in the reading seems to toggle on and off at each reset and while at first I thought it was bouncing from one board to the other, I then saw that at times the voltages match likely because both errors are on the same leg. In other words, it is something that somewhat randomly affects one leg (voltage measurements of L1 and L2) or the other with it often switching which one it affects. Hopefully this makes sense... To be clear, my setup has not been installed yet as once I do that I won't be able to troubleshoot further. Both transformers are connected to the same outlet and both voltage inputs have been calibrated. When the error is not affecting the channel, the reading is correct. Both channels read the voltage correctly unless the error is affecting that channel.

If we look at the lower level code that implements this board / chip, is there any code that could be randomly affecting just one of the 2 voltage measurement channels, or maybe both or neither, somewhat randomly?

alexruffell commented 1 year ago

I made a simple button where I can run IC setup any time:

Where did you define the chip ids? I don't have any chip related ids in my YAML. I'd like to test what you did to see if I can get the error to bounce around.

EDIT: I added id: chip# right after each platform for the 4 chips. I then added the setup command for the 2 additional chips.

image

My starting point shows that L2 and L2 AO match and that L1 and L1 AO match. The chip setup appears to make no difference (assuming I implemented it correctly). At 11:21 I hit the reset button and L1 error went away and it now matches the other 3 leaving just L1 AO with the 0.5V error. At 11:26 I hit reset again and L1 got the error back...

image
Cougar commented 1 year ago

Here you found the same behavior that I see. Chip reset via setup hook doesn't chance anything but reset button or ESPHome restart does. Only thing that happens in latter case is the I2C setup and the data collection timing relative to AC zero crossing is probably different. Could any of them be a reason why the difference changes?

alexruffell commented 11 months ago

I built a little calibration rig to try to get to the bottom of this issue. While I understand that the voltage difference is minimal and in the big scheme of things irrelevant, I wonder whether there is some way to fix it given this seems to be a software issue. I implemented a faster SPI clock (1MHz) courtesy of @descipher and while I have not noticed any drawbacks, I am inclined to think it helps given this time around I was able to narrow the gap between the voltages reported by the 2 boards. The one thing that still bugs me is why does L1 voltage reported by the main board and the addon board swap places reporting a slightly higher value nearly each time the ESP is re-flashed?

The graph below only shows L1 V and L1 AO V so it is easier to see how they swap places. I have a larger transformer feeding L1 on both boards, and a smaller transformer feeding L2 on both boards. Both transformers are currently being powered by the same mains connection (outlet) and I am monitoring the voltage with a benchtop 5 1/2 digit DMM. I am not trying to obtain absolute accuracy, but rather the 4 measurements to be as close as possible to each other.

In this thread there is discussion of the effect of the ESP power draw when using WIFI, but that would not explain why L1V and L1AOV trade places... which is the issue I am trying to fix.

image

I am also wondering whether the 4 voltage readings are actually necessary to read 6 power measurements. Would it be possible to just use the L1 and L2 voltages measurements from one of the two boards for the purposes of the entire system? I am guessing not as each chip will independently read voltage & current and calculate power by multiplying the two?

To calibrate the system I am using a 150W 120V bulb (my load) and power it through a 4 x loop so the power that the current transformers see it 4x which, given bulb wattage tolerances, translates to 4A. The clamp in the picture is what I use to read the current (reading it typically around 4.017A). The caps I soldered on the ESP are just tests to see if I can reduce fluctuations caused by the ESP draw but have no effect on what I am trying to fix.

image
descipher commented 11 months ago

I see a number of potential reasons for the variation in L1, L2 or L3 voltages.

1) UoffsetA/B/C in not implemented in the code and thus the variation of L1, L2 or L3 voltage circuit value/noise differences cannot be calibrated. To correctly calibrate the input to each chip both UgainA/B/C and UoffsetA/B/C must be implemented. Gain is not the same as offset. Gain is the factor and offset is the position to factor. Both are required to have accuracy.

See: image

2) The value tolerance of the quad input resistor pack R24 on the 6 channel board is 5% which could have enough difference to be observable. The other circuit resistors are not visable on the published diagram so not sure what those tolerences are. image

3) With all the leads/connectors/EMI there is a high probability of inductive/noise/impedence variables which require full calibration to keep the input measurement in spec.

4) The input sampling is also too low, a single read is not very accurate, several samples should be read and averaged.

descipher commented 11 months ago

Sorry, needed to include this: image

alexruffell commented 11 months ago

@descipher - I realize this board, and likely the chip, are not meant for high accuracy applications and am ok with some error. Also, not all of my equipment is calibrated or at least not recently so my end goal is mostly to make the readings jive for the lack of a better word. There is a parameter I can change to "calibrate" the voltage measurements and I can get only so close because L1 on the 2 boards appear to keep swapping an error that skews the readings.

If you look at the graph in my last post, you will notice that the top trace starts green, then after a reboot trades its place with blue, and then back to green. It doesn't happen at every single reboot, and in fact I point out 3 reboots but the error moved only twice. When this error moves from one input to the other, it messes with my calibration so I find myself having to correct it in the other direction which then gets nullified when it swaps again. Since this swap happens pretty reliably, I am hoping it is a software implementation issue that can be addressed. Resistors with high tolerance values such as the 5% one you mentioned would affect the readings but not cause the error to move from one board to the other, right?

With all the leads/connectors/EMI there is a high probability of inductive/noise/impedence variables which require full >calibration to keep the input measurement in spec.

The best I can do to mitigate this issue is what I show in the picture. The CTs are labeled for the respective circuit branches they will measure and the black box with numbered 3.5mm connectors will be inside the outdoor electrical panel. The gray wires will exit the panel and enter the box you see in the picture through electrical conduit. In other words, I am trying to do a "system calibration" in my office as if it were installed because I won't be able to do one once it is installed outdoors. The gray cables will be shorter but I doubt the reduced resistance will have a significant impact.

When the ESP32 reboots, something happens to the main voltage input on one of the two boards causing them to diverge. In rare occasions whatever happens makes then "agree". Your 1MHz SPI tweak seems to have had an impact as I was never able to get the V readings so close, but many other things may have changed too, so I am not sure.

The input sampling is also too low, a single read is not very accurate, several samples should be read and averaged.

I was planning on implementing a sliding_window_moving_average filter in ESPHome as a way to reduce the impact of outlier readings but those will also be hidden by reducing the decimals to just 1 position once I did all I can do to calibrate this system.

UoffsetA/B/C in not implemented in the code and thus the variation of L1, L2 or L3 voltage circuit value/noise differences >cannot be calibrated. To correctly calibrate the input to each chip both UgainA/B/C and UoffsetA/B/C must be implemented. Gain is not the same as offset. Gain is the factor and offset is the position to factor. Both are required to have accuracy.

This sounds like something that can be improved in the custom component that supports this chip in ESPHome. I don't have the ability to do so myself. Hopefully someone will contribute it in the future, however I am also guessing that it would not help with the error jumping from one board to the other randomly.

Below is the latest revision of the YAML I am using. The update rate is set to 3s only to make it easier to calibrate the measurements. As you mentioned in another thread I typically would keep it at a higher value but not so high I can't see a relevant change fast enough for it to be useful in troubleshooting.

In regards to the snapshot on Offsets, aren't those what I am changing? See my substitutions section.

# Example config for when jp8-jp11 are all bridged - this connects all the voltage channels and allows for power to be calculated by the meter directly.
# Boards >= v1.4 jp8-jp11 are removed and have all voltage channels connected

# Oven 40A, Dryer 30A, HVAC 25A, HVAC 40A, SubPanel 80A

substitutions:
  devicename: home-energy-meter
  devicename_no_dashes: home_energy_meter
  friendly_devicename: "Home Energy Meter"
  device_description: "Home Energy Meter"
  appliance1: "Dryer"
  appliance2: "Oven"
#  appliance3: "TBD"
  hvac1: "Downstairs HVAC"
  hvac2: "Upstairs HVAC"
# Current Transformers:
# 20A/25mA SCT-006: 11143
# 30A/1V SCT-013-030: 8650
# 50A/1V SCT-013-050: 15420
# 80A/26.6mA SCT-010: 41660
# 100A/50ma SCT-013-000: 27518
# 120A/40mA: SCT-016: 41787
# 200A/100mA SCT-024: 27518
  current_cal_ct1:  '46800' #L1 Main
  current_cal_ct2:  '42480' #L1 Dryer
  current_cal_ct3:  '42480' #L1 Oven
  current_cal_ct4:  '42565' #L2 Oven was 42075
  current_cal_ct5:  '42580' #L2 Dryer
  current_cal_ct6:  '47010' #L2 Main
  current_cal_ct7:  '41660' #L1 CT7
  current_cal_ct8:  '42355' #L1 HVAC DS
  current_cal_ct9:  '42265' #L1 HVAC US
  current_cal_ct10: '42375' #L2 HVAC US
  current_cal_ct11: '42365' #L2 HVAC DS
  current_cal_ct12: '41660' #L2 CT12
# voltage calibrations done by feeding 60Hz sine on seconday input
# and painfully on primary but they match as close as I can get it
  v1_cal: '5018' #5022 1L1
  v2_cal: '5388' #5378 1L2      V2 and V4 match when stable 60Hz sine is fed to them. Do not change independently
  v3_cal: '4998' #5004 2L1 (AO)
  v4_cal: '5362' #5352 2L2 (AO) V2 and V4 match when stable 60Hz sine is fed to them. Do not change independently
  # Interval of how often the wifi info is updated
  update_interval_wifi: "120s"
  # Interval of how often the power is updated
  update_interval_s:   "3s"
  update_interval_s_totals:   "60s"
  v_accuracy: "3"
  a_accuracy: "3"
  w_accuracy: "3"
  kwh_accuracy: "1"

esphome:
  name: ${devicename}
  friendly_name: "${friendly_devicename}"
  comment: ${device_description}
  platform: ESP32
  board: nodemcu-32s

external_components:
  source: github://descipher/esphome@component.atm90e32
  components: [ atm90e32 ]
  refresh: 0s

wifi:
  ssid: !secret iot_wifi_ssid
  password: !secret iot_wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "${devicename}"
    password: !secret iot_wifi_password

#Faster than DHCP. Also use if can't reach because of name change
  manual_ip:
    static_ip: 192.168.3.198
    gateway: 192.168.3.1
    subnet: 255.255.255.0
    dns1: 192.168.1.25
    dns2: 192.168.1.36

#Manually override what address to use to connect to the ESP.
#Defaults to auto-generated value. Example, if you have changed your
#static IP and want to flash OTA to the previously configured IP address.
  #use_address: 192.168.3.198

# Enable logging
logger:
  baud_rate: 921600

# Enable Home Assistant API
api: 
#  password: !secret api_pwd

ota:
  password: !secret ota_pwd

web_server:
  port: 80

# Sync time with Home Assistant
time:
  - platform: homeassistant
    id: ha_time

spi:
  clk_pin: 18
  miso_pin: 19
  mosi_pin: 23

text_sensor:
  - platform: wifi_info
    ip_address:
      name: "IP"
      icon: "mdi:ip-outline"
      update_interval: ${update_interval_wifi}
    ssid:
      name: "SSID"
      icon: "mdi:wifi-settings"
      update_interval: ${update_interval_wifi}
    bssid:
      name: "BSSID"
      icon: "mdi:wifi-settings"
      update_interval: ${update_interval_wifi}
    mac_address:
      name: "MAC"
      icon: "mdi:network-outline"
    scan_results:
      name: "Wifi Scan"
      icon: "mdi:wifi-refresh"
      disabled_by_default: true

sensor:
  - platform: wifi_signal
    name: "WiFi Signal"
    update_interval: ${update_interval_wifi}
    device_class: signal_strength

#IC1
  - platform: atm90e32
    id: chip1
    cs_pin: 5
    phase_a:
      voltage:
        name: "L1 V"
        id: ic1Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "L1 A"
        id: ct1Amps
        accuracy_decimals: ${a_accuracy}
        # The max value for current that the meter can output is 65.535. If you expect to measure current over 65A, 
        # divide the gain_ct by 2 (120A CT) or 4 (200A CT) and multiply the current and power values by 2 or 4 by uncommenting the filter below
        filters:
          - multiply: 4
      power:
        name: "L1 W"
        id: ct1Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - multiply: 4
          #With no load why do I have to filter out negative W values??
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct1}
    phase_b:
      current:
        name: "${appliance1} L1 A" #Dryer
        id: ct2Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance1} L1 W" #Dryer
        id: ct2Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct2}
    phase_c:
      current:
        name: "${appliance2} L1 A" #Oven
        id: ct3Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance2} L1 W" #Oven
        id: ct3Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct3}
    frequency:
      name: "L1 Hz"
      device_class: frequency
    chip_temperature:
      name: "L1 Chip Temperature"
      id: l1_chip_temperature
      device_class: temperature
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}
#IC2
  - platform: atm90e32
    id: chip2
    cs_pin: 4
    phase_a:
      current:
        name: "${appliance2} L2 A" #Oven
        id: ct4Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance2} L2 W" #Oven
        id: ct4Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct4}
    phase_b:
      current:
        name: "${appliance1} L2 A" #Dryer
        id: ct5Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance1} L2 W" #Dryer
        id: ct5Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct5}
    phase_c:
      #this voltage is only needed if monitoring 2 voltages
      voltage:
        name: "L2 V"
        id: ic2Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "L2 A"
        id: ct6Amps
        accuracy_decimals: ${a_accuracy}
        # The max value for current that the meter can output is 65.535. If you expect to measure current over 65A, 
        # divide the gain_ct by 2 (120A CT) or 4 (200A CT) and multiply the current and power values by 2 or 4 by uncommenting the filter below
        filters:
          - multiply: 4
      power:
        name: "L2 W"
        id: ct6Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - multiply: 4
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct6}
    frequency:
      name: "L2 Hz"
      device_class: frequency
    chip_temperature:
      name: "L2 Chip Temperature"
      id: l2_chip_temperature
      device_class: temperature
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}

#IC1 AddOn
  - platform: atm90e32
    id: chip3
    cs_pin: 0
    phase_a:
      voltage:
        name: "L1 AO V"
        id: ic3Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "CT7 Amps"
        id: ct7Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "CT7 Watts"
        id: ct7Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct7}
    phase_b:
      current:
        name: "${hvac1} L1 A" #Downstairs
        id: ct8Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac1} L1 W" #Downstairs
        id: ct8Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct8}
    phase_c:
      current:
        name: "${hvac2} L1 A" #Upstairs
        id: ct9Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac2} L1 W" #Upstairs
        id: ct9Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct9}
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}
#IC2 AddOn
  - platform: atm90e32
    id: chip4
    cs_pin: 16
    phase_a:
      current:
        name: "${hvac2} L2 A" #Upstairs
        id: ct10Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac2} L2 W" #Upstairs
        id: ct10Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct10}
    phase_b:
      current:
        name: "${hvac1} L2 A" #Downstairs
        id: ct11Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac1} L2 W" #Downstairs
        id: ct11Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct11}
    phase_c:
      voltage:
        name: "L2 AO V"
        id: ic4Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "CT12 Amps"
        id: ct12Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "CT12 Watts"
        id: ct12Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct12}
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}

#Appliance1 Watts (Dryer)
  - platform: template
    name: "${appliance1} W" #Dryer
    id: appliance1TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct2Watts).state + id(ct5Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#Appliance1 Amps (Dryer)   
  - platform: template
    name: "${appliance1} A" #Dryer
    id: appliance1TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct2Amps).state + id(ct5Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals} 

#Appliance1 kWh - (Dryer)
  - platform: total_daily_energy
    name: "${appliance1} kWh" #Dryer
    power_id: appliance1TotalWatts
    accuracy_decimals: ${w_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#Appliance2 Total Watts (Oven)
  - platform: template
    name: "${appliance2} W" #Oven
    id: appliance2TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct3Watts).state + id(ct4Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#Appliance2 Total Amps (Oven)
  - platform: template
    name: "${appliance2} A" #Oven
    id: appliance2TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct3Amps).state + id(ct4Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals}

#Appliance2 kWh - (Oven)
  - platform: total_daily_energy
    name: "${appliance2} kWh" #Oven
    power_id: appliance2TotalWatts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#HVAC1 Watts (Downstairs)
  - platform: template
    name: "${hvac1} W" #Downstairs
    id: hvac1TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct8Watts).state + id(ct11Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#HVAC1 Amps (Downstairs)  
  - platform: template
    name: "${hvac1} A" #Downstairs
    id: hvac1TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct8Amps).state + id(ct11Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals} 

#HVAC1 kWh (Downstairs)
  - platform: total_daily_energy
    name: "${hvac1} kWh" #Downstairs
    power_id: hvac1TotalWatts
    accuracy_decimals: ${w_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#HVAC2 Watts (Upstairs)
  - platform: template
    name: "${hvac2} W" #Upstairs
    id: hvac2TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct9Watts).state + id(ct10Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#HVAC2 Amps (Upstairs)  
  - platform: template
    name: "${hvac2} A" #Upstairs
    id: hvac2TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct9Amps).state + id(ct10Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals} 

#HVAC2 kWh (Upstairs)
  - platform: total_daily_energy
    name: "${hvac2} kWh" #Upstairs
    power_id: hvac2TotalWatts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#Watts - L1 & L2
  - platform: template
    name: "L1 & L2 W"
    id: l1l2Watts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct1Watts).state + id(ct6Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#Amps - L1 & L2
  - platform: template
    name: "L1 & L2 A"
    id: l1l2Amps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct1Amps).state + id(ct6Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals}  

#Volts - L1 & L2 - Average
  - platform: template
    name: "L1 & L2 V (Avg)"
    id: l1l2Volts
    accuracy_decimals: ${v_accuracy}
    lambda: return (id(ic1Volts).state + id(ic2Volts).state)/2;
    unit_of_measurement: "V"
    device_class: voltage
    update_interval: ${update_interval_s_totals}  

#kWh - L1
  - platform: total_daily_energy
    name: "L1 kWh"
    power_id: ct1Watts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#kWh - L2
  - platform: total_daily_energy
    name: "L2 kWh"
    power_id: ct6Watts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#kWh - Whole House (L1 + L2)
  - platform: total_daily_energy
    name: "L1 & L2 kWh"
    power_id: l1l2Watts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

button:
  - platform: safe_mode
    name: "Restart (Safe Mode)"
    entity_category: diagnostic

  - platform: restart
    name: "Restart"
    entity_category: diagnostic

  - platform: template
    name: "IC Setup"
    on_press:
      then:
        lambda: !lambda |-
          id(chip1).setup();
          id(chip2).setup();
          id(chip3).setup();
          id(chip4).setup();
descipher commented 11 months ago

@alexruffell

There are some code issues that I suspect need to be addressed.

The code does a SoftReset reg write and fails to wait the required time b4 writing out other reg values.

  this->write16_(ATM90E32_REGISTER_SOFTRESET, 0x789A);    // Perform soft reset
  this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA);  // enable register config access
  this->write16_(ATM90E32_REGISTER_METEREN, 0x0001);      // Enable Metering
  if (this->read16_(ATM90E32_REGISTER_LASTSPIDATA) != 0x0001) {
    ESP_LOGW(TAG, "Could not initialize ATM90E32 IC, check SPI settings");
    this->mark_failed();
    return;
  }

There are no checks to see if the subsequent write is ack'd after the reset. This can be why you are seeing differing results. The ATM90E32 has a verification method where all that is needed is a compare to the LastSPIData and the Buffer value. This should be changed so that the next write operation is verified after reset and it needs to wait 5ms + 1ms which is the minimum + 1 which applieas to a powered on state.

image

I have increased the SPI clock to 2MHz and see no issues.

I have also added the delay and check in the code of my branch named component.atm90e32

  this->write16_(ATM90E32_REGISTER_SOFTRESET, 0x789A);    // Perform soft reset
  delay(6);                                               // Wait the minimum time after reset
  this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA);  // enable register config access
    if (this->read16_(ATM90E32_REGISTER_LASTSPIDATA) != 0x55AA) {
    ESP_LOGW(TAG, "Could not initialize ATM90E32 IC, check SPI settings");
    this->mark_failed();
    return;
  }
alexruffell commented 11 months ago

@descipher I reflashed my system to pull in your latest changes and this is what happened:

image

The first red arrow is when I reflashed it but left the update rate at 10s. The second arrow was when I reflashed it after reducing the update rate to 3s.

I let some time go by and reflashed it again without making any changes. The gap between L1 and L2 remained, but blue and green swapped places again (see graph below). The line that is closest to the actual value is the bottom one when only considering blue and green.

The gap may be a result of my previous calibration which may need to be adjusted after your changes, however the two L1 seem to still be trading places. [Edit: this seems to be disproved by the latest graph where all measurements match]

I reflashed the ESP again... and this time something interesting happened... Green (L1 V) now jives with L2 and L2 AO as it should given they are all measuring the same thing. The error on blue seems to have remained the same.

image

Flashed it again and green joined blue again...

image

I rebooted the system many times to see if I could trigger the issue but it doesn't seem to have the same effect of a reflash (without changes). I reflashed the system a few times and the system is working properly (only to revert to the error at the next flash as this is not the first time it happens):

image

More granular view showing the desired behavior (!!). I can't say that rebooting it a random number of times would not produce this.

image

One thing I have not mentioned, is that the two transformers that provide the low AC voltage representing line voltage are different as one has a nominal output of 9V and the other 12V (actual measured it quite a bit higher). Just mentioning it for completeness. Regardless, this lates graph shows the "expected" and "desired" behavior so I am guessing my setup is ok and that the issue is probably still in software.

Edit: A while later... a reboot broke the party:

image

Edit2: Many reboots later, I see the issue happening and I often see all of the measurements "jive" right after the reboot, and then the error appears on the next sample:

image

The error often appears to be somewhere around 0.5V although it is hard to determine what the error actually is. What I am calling error is the rough difference this is mostly just the delta between the bunch and the outlier as shown here:

image
descipher commented 11 months ago

Need to review the application guide. 2MHz may need a delay in some specific operations. I see no issues on a single chip running an esp8266. Your Esp32 is faster and that will make a small difference.

alexruffell commented 11 months ago

Also, the issue I am seeing is across 2 boards with a total of 4 atm90e32 chips which may be another reason you are not seeing it. Maybe the issue lies with whatever happens when the 4 chips are setup/sync'd.

descipher commented 11 months ago

One of the thoughts I have surrounds the possibility that we are seeing a gain uniformity problem. We will need to supply the same rms source to all the voltage inputs in order to verify that type of problem. The reason I suspect that is because there appears to be an ADC resolution threshold in which we see a step up in value for a measurements. Averaging would help somewhat but it would not fix it. So yes the 12v vs 9v inputs are related to that type of issue. Try setting the gain equal for all voltage inputs and see is what it reads. This will also help see if there is uniformity.

descipher commented 11 months ago

Added the voltage offset calibration init to my repo.

CircuitSetup commented 11 months ago

Added the voltage offset calibration init to my repo.

Excellent work!

I originally wrote something to calibrate the offset registers for the arduino library, but it was never carried over to the esphome sensor.

descipher commented 11 months ago

Added the voltage offset calibration init to my repo.

Excellent work!

I originally wrote something to calibrate the offset registers for the arduino library, but it was never carried over to the esphome sensor.

Thanks, Also just added the current offset calibration and applied some averaging to the voltage reads to see if we get better stability.

alexruffell commented 11 months ago

I am now using the same 12V AC 1A transformer for all voltage inputs. Before readjusting the calibration values, I was now reading 119V and 127V so 8V AC difference.

My calibration settings changed as follows:

#WHEN USING 2 TRANSFORMERS
  # v1_cal: '5022' #green
  # v2_cal: '5378' #yellow
  # v3_cal: '5004' #blue
  # v4_cal: '5352' #orange
#FOR TESTING ONLY - USING SINGLE TRANSFORMER
  v1_cal: '5020' #green
  v2_cal: '5028' #yellow
  v3_cal: '5002' #blue
  v4_cal: '5002' #orange

The bottom 3 tend to jive well with my 5 1/2 digit DMM, and as before, blue and green trade places upon a reflash with no changes made. It took 3 back to back reboots to make it happen again so it seems a bit more repeatable with reflashes.

image

After multiple reboots, the distribution of the 4 voltage measurements seems to have widened as if the error can affect other inputs like L2 AO V (orange) but in the opposite direction.

A reflash w/o changes confirmed it:

image

Note that yellow and orange jive again... and green and blue traded places.

@descipher - Do I have to add any YAML to take advantage of any of the new settings (offsets, etc) that you have implemented?

I wish I could eliminate the line fluctuations (this morning while doing this, it constantly fluctuated between 118 and 120V). Just makes it a bit harder to observe the effects of everything else I am doing.

EDIT: It always seems that the 2nd trace is the closest to the actual value. It am trying to get the others to get closer but it seems like a change made to one affects the others - just hard to tell with the constant fluctuations.

descipher commented 11 months ago

That tells me it not a gain unity issue. Is that flashed currently? The calibration does not require input control. It just reads the voltage 5 times averages it the calculates the two’s compliment as described in the ref. I am going to check with Atmel on the calibration, something does not look right to me.

alexruffell commented 11 months ago

That tells me it not a gain unity issue. Is that flashed currently?

Not sure I understand what you are referring to. I am assuming that whenever I recompile I am pulling in your custom component. given I see this at the top of my logs:

INFO ESPHome 2023.7.1
INFO Reading configuration /config/esphome/home-energy-meter.yaml...
INFO Updating https://github.com/descipher/esphome.git@component.atm90e32

I really appreciate all the troubleshooting you are doing with me!!

descipher commented 11 months ago

The refresh value of 0 will force an update on every compile. Your welcome.

descipher commented 11 months ago

I believe I have an answer to your calibration problem. Looking closely at your yaml config I can see issues. Each board has two chips and the voltage is fed into separate circuits from the two AC tranformer rms references. With a single AC transformer as a reference it would require three different gain settings on each chip using the one AC rms reference. In this case they are all the same. (e.g. v1_cal is set on phase a,b,c) Each chips phase voltage inputs have different resistor networks and each one can vary up to 5% Now considering we have L1 and L2 measurements accross phase a,b and c then we need to give them all separate voltage gain calibration values and not use only the 4 defined as substitutions. The circuits are all slightly different and thats why we are seeing a small differences when 4 values are applied.

substitutions:
  v1_cal: '5018' #5022 1L1
  v2_cal: '5388' #5378 1L2      V2 and V4 match when stable 60Hz sine is fed to them. Do not change independently
  v3_cal: '4998' #5004 2L1 (AO)
  v4_cal: '5362' #5352 2L2 (AO) V2 and V4 match when stable 60Hz sine is fed to them. Do not change independently

This is the method Atmel uses to calculate the voltage gain for each independant input circuits gain register.

Example: Voltage gain calibration
Assume:
The initial value of phase A voltage gain register UgainA is 8000H (32768)
Reference meter output voltage is 220.00V
Voltage rms register readout UrmsA = 3039H (12345)
The higher 8 bits of voltage LSB register readout UrmsALSB = 43H (67)
Thus:
voltage measured value = (UrmsA x 0.01) + (UrmsALSB x 0.01 / 256)
 = (12345 x 0.01) + (67 x 0.01 / 256)
 =123.453 V
voltage gain = 220.00 / 123.453 x 32768 = 58395 = 0E41BH
So the register can be set to:
UgainA = 0E41BH 
Cougar commented 11 months ago

All this calibration thing is true for a constant difference between readings but there should be something else that changes after (each) restart.

If you look my https://github.com/CircuitSetup/Expandable-6-Channel-ESP32-Energy-Meter/issues/88#issuecomment-1382315333 you see how relatively stable is the difference between resets.

I haven't played with that meantime but what I finally found was that just running chip reset code itself didn't change anything but whole ESPHome restart did (flashing also does reset the controller). I'm not sure if SPI init could make any difference.

Anyway, it is good to know that I'm not the only one seeing this and maybe we can figure out what is going on in this chip.

descipher commented 11 months ago

All this calibration thing is true for a constant difference between readings but there should be something else that changes after (each) restart.

If you look my #88 (comment) you see how relatively stable is the difference between resets.

I haven't played with that meantime but what I finally found was that just running chip reset code itself didn't change anything but whole ESPHome restart did (flashing also does reset the controller). I'm not sure if SPI init could make any difference.

Anyway, it is good to know that I'm not the only one seeing this and maybe we can figure out what is going on in this chip.

It's possible that we are observing an order of init anomaly due to the lambda's. These are callbacks that are fired almost simutaneouly. Depending on the SPI reply time each will have a possiblility or order variation. This may be the random change element that skews the measurement order. This could be handled by the python init as a solution to the possible order variations.

  - platform: template
    name: "IC Setup"
    on_press:
      then:
        lambda: !lambda |-
          id(chip1).setup();
          id(chip2).setup();
          id(chip3).setup();
          id(chip4).setup(); 
alexruffell commented 11 months ago

@descipher The last code snippet was just me trying to implement a reset of the 4 chips by pressing a button. Something @Cougar was discussing in prior posts but it doesn't appear to trigger the issue I am seeing. To avoid confusion, I will remove that code from my YAML.

descipher commented 11 months ago

Ok, sorry miss read on that lambda . Can you try setting the 4 specific phase voltage inputs to their respective gains without the substitutions. e.g. only directly set the voltage gain for the phase x point where you have the AC input connected and sending out to HA. This way we know exactly what resistor network values we are actually reading from, all others would be out of spec from the calibration and immediatly observed as abnormal.

yaml would be directly set.

phase a:
  voltage:
    name: "L1 V" 
      id: icxVolts
      accuracy_decimals: ${v_accuracy}
  gain_voltage: 5000
  gain_ct: ${current_cal_ctx}
CircuitSetup commented 11 months ago

With a single AC transformer as a reference it would require three different gain settings on each chip using the one AC rms reference. In this case they are all the same. (e.g. v1_cal is set on phase a,b,c) Each chips phase voltage inputs have different resistor networks and each one can vary up to 5%

There shouldn't be any difference between the 3 voltage channels per meter chip because they are all tied together off of the same resistor dividers. Additionally, any boards made in the past couple years have 1% resistor arrays, and 0.1% resistor dividers (and burden resistors).

The only thing that would be causing any variance between the voltage channels is the timing of the registers being read, Averaging multiple readings should solve that issue. That being said, the power calculations are done internally, and if you had a current transformer measuring the same load over the same amount of time, the accumulated watt hours would be the same.

descipher commented 11 months ago

With a single AC transformer as a reference it would require three different gain settings on each chip using the one AC rms reference. In this case they are all the same. (e.g. v1_cal is set on phase a,b,c) Each chips phase voltage inputs have different resistor networks and each one can vary up to 5%

There shouldn't be any difference between the 3 voltage channels per meter chip because they are all tied together off of the same resistor dividers. Additionally, any boards made in the past couple years have 1% resistor arrays, and 0.1% resistor dividers (and burden resistors).

The only thing that would be causing any variance between the voltage channels is the timing of the registers being read, Averaging multiple readings should solve that issue. That being said, the power calculations are done internally, and if you had a current transformer measuring the same load over the same amount of time, the accumulated watt hours would be the same.

@CircuitSetup Thanks for the clarification on the resistor tollerance, I assume this is a 1.4 rev 1 board, based on the circuit for the 1.4 rev1 board you published we have a 20:1 divider into VAP and VAN on IC1 which is connected to phase A,B,C or VAP2 and VAN2 to IC2 phase A,B,C. I see @alexruffell has a configuration is set for L1 to VAP and VAN sourced from phase_a and L2 to VAP2 and VAN2 which is sourced from phase _c. Since all phases are connected then only the divider for one chip would have any variance and since its a 4 pack they should be extremely close in value. But two boards will have separate packs with up to 1% variation and based on the datasheet we could see up to 0.01v RMS which will be calc up to plus or minus 0.5v on the HA send.

The code updates on my repo are currently doing averaging over 10 consecutive reads so thats not the issue. What I need to see is the result of the gain setting directly on the expected phase to see if there is a reading from an unexpected source outside of what is targeted. That will show up immediatedly if the gain is only set on the used target inputs.

alexruffell commented 11 months ago

@descipher - My house is powered by a residential 240V 200A split phase supply. As far as I recall, each board has 3 channels on one phase and 3 on another phase therefore when I install this, I need to ensure the CTs are installed on the matching phase.

Before I rewired to feed both boards from the same transformer for testing purposes, I had a 12V 14VA transformer feed the main voltage input for both boards, and a 9V 1.5VA feed the secondary voltage input for both boards. I am looking into buying 2 identical encapsulated transformers to reduce variables but am struggling with finding one powerful enough (for ESP and to avoid excessive fluctuations during TX) but also small enough to fit on the PCB.

I did not write the original YAML so I don't know why it was setup as it is. I modified it quite a bit but not the core structure. I figured the 4 voltage calibrations were needed because each board has an input that measures voltage from both transformers (one per phase). Therefore L1 V and L1 AO V are from the same phase/transformer, and the same applies for L2 V and L2 AO V (AO is the add-on board). From all my tinkering I seem to be able to get "close enough" readings for voltage measurements on the same phase (ie: L1V & L1 AO V) but have trouble getting L1 and L2 voltages closer.

Anyhow, if I persistently reflash / reboot the ESP32 without code changes, on occasion all 4 voltage inputs get much closer as discussed in a prior post. The graph that shows it is this one (see !!): image

While improving overall accuracy is great, I think that eliminating that "error" that occasionally affects x number of inputs is more important. Why would one or more inputs measure voltages close to actual only to skew away ~0.5V up or down after a reflash? Tolerances, design imperfections, etc would introduce error that would likely be constant or change slowly, but what I observe is a sudden change that pushes an otherwise calibrated channel off by ~0.5V.

I am glad to make any changes you require but am unsure I understood your instructions. Below is a simplified version of the YAML where the vn_cal are used.

Main Board IC1 phase a, b, c use v1_cal IC2 phase a, b, c use v2_cal

Addon Board IC1 phase a, b, c use v3_cal IC2 phase a, b, c use v4_cal

#IC1
  - platform: atm90e32
    id: chip1
    cs_pin: 5
    phase_a:
      voltage:
        name: "L1 V"
        id: ic1Volts
      current:
        name: "L1 A"
        id: ct1Amps
        # The max value for current that the meter can output is 65.535. If you expect to measure current over 65A, 
        # divide the gain_ct by 2 (120A CT) or 4 (200A CT) and multiply the current and power values by 2 or 4
        filters:
          - multiply: 4
      power:
        name: "L1 W"
        id: ct1Watts
        filters:
          - multiply: 4
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct1}
    phase_b:
      current:
        name: "${appliance1} L1 A" #Dryer
        id: ct2Amps
      power:
        name: "${appliance1} L1 W" #Dryer
        id: ct2Watts
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct2}
    phase_c:
      current:
        name: "${appliance2} L1 A" #Oven
        id: ct3Amps
      power:
        name: "${appliance2} L1 W" #Oven
        id: ct3Watts
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct3}
    frequency:
      name: "L1 Hz"
      device_class: frequency
    line_frequency: 60Hz
    gain_pga: 1X

#IC2
  - platform: atm90e32
    id: chip2
    cs_pin: 4
    phase_a:
      current:
        name: "${appliance2} L2 A" #Oven
        id: ct4Amps
      power:
        name: "${appliance2} L2 W" #Oven
        id: ct4Watts
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct4}
    phase_b:
      current:
        name: "${appliance1} L2 A" #Dryer
        id: ct5Amps
      power:
        name: "${appliance1} L2 W" #Dryer
        id: ct5Watts
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct5}
    phase_c:
      #this voltage is only needed if monitoring 2 voltages
      voltage:
        name: "L2 V"
        id: ic2Volts
      current:
        name: "L2 A"
        id: ct6Amps
        # The max value for current that the meter can output is 65.535. If you expect to measure current over 65A, 
        # divide the gain_ct by 2 (120A CT) or 4 (200A CT) and multiply the current and power values by 2 or 4
        filters:
          - multiply: 4
      power:
        name: "L2 W"
        id: ct6Watts
        filters:
          - multiply: 4
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct6}
    frequency:
      name: "L2 Hz"
      device_class: frequency
    line_frequency: 60Hz
    gain_pga: 1X

#IC1 AddOn
  - platform: atm90e32
    id: chip3
    cs_pin: 0
    phase_a:
      voltage:
        name: "L1 AO V"
        id: ic3Volts
      current:
        name: "CT7 Amps"
        id: ct7Amps
      power:
        name: "CT7 Watts"
        id: ct7Watts
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct7}
    phase_b:
      current:
        name: "${hvac1} L1 A" #Downstairs
        id: ct8Amps
      power:
        name: "${hvac1} L1 W" #Downstairs
        id: ct8Watts
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct8}
    phase_c:
      current:
        name: "${hvac2} L1 A" #Upstairs
        id: ct9Amps
      power:
        name: "${hvac2} L1 W" #Upstairs
        id: ct9Watts
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct9}
    line_frequency: 60Hz
    gain_pga: 1X

#IC2 AddOn
  - platform: atm90e32
    id: chip4
    cs_pin: 16
    phase_a:
      current:
        name: "${hvac2} L2 A" #Upstairs
        id: ct10Amps
      power:
        name: "${hvac2} L2 W" #Upstairs
        id: ct10Watts
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct10}
    phase_b:
      current:
        name: "${hvac1} L2 A" #Downstairs
        id: ct11Amps
      power:
        name: "${hvac1} L2 W" #Downstairs
        id: ct11Watts
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct11}
    phase_c:
      voltage:
        name: "L2 AO V"
        id: ic4Volts
      current:
        name: "CT12 Amps"
        id: ct12Amps
      power:
        name: "CT12 Watts"
        id: ct12Watts
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct12}
    line_frequency: 60Hz
    gain_pga: 1X
CircuitSetup commented 11 months ago

@CircuitSetup Thanks for the clarification on the resistor tollerance, I assume this is a 1.4 rev 1 board, based on the circuit for the 1.4 rev1 board you published we have a 20:1 divider into VAP and VAN on IC1 which is connected to phase A,B,C or VAP2 and VAN2 to IC2 phase A,B,C. I see @alexruffell has a configuration is set for L1 to VAP and VAN sourced from phase_a and L2 to VAP2 and VAN2 which is sourced from phase _c. Since all phases are connected then only the divider for one chip would have any variance and since its a 4 pack they should be extremely close in value. But two boards will have separate packs with up to 1% variation and based on the datasheet we could see up to 0.01v RMS which will be calc up to plus or minus 0.5v on the HA send.

That is correct. It's unclear on the datasheet whether the plus minus 0.5% variance is between ADC channels on a single chip, or from chip to chip. It just says "less than ±0.5% fiducial error for Vrms, Irms, mean active/ reactive/ apparent power, frequency, power factor and phase angle."

The code updates on my repo are currently doing averaging over 10 consecutive reads so thats not the issue. What I need to see is the result of the gain setting directly on the expected phase to see if there is a reading from an unexpected source outside of what is targeted. That will show up immediatedly if the gain is only set on the used target inputs.

One other thing to consider is that VAP/VAN also go into a rectifier/capacitor before going into the SMPS. The capacitor distorts the AC waveform a bit when charging up. This normally isn't a problem as long as the waveform's not distorted to begin with.

Also, I'm not sure if this matters or not since I haven't tested it, but the Application Note indicates the offset calibration should be done before the gain calibration. I believe you have the offset running after the gain in your code.

atm90e32_calibration_flow

CircuitSetup commented 11 months ago

I figured the 4 voltage calibrations were needed because each board has an input that measures voltage from both transformers

Yes, that is correct if you want the highest level of accuracy.

Why would one or more inputs measure voltages close to actual only to skew away ~0.5V up or down after a reflash?

That I still can't figure out. I thought the offset calibration with averages would correct this. Now I'm wondering if it's just a matter of which calibration code is firing first. Have you tried moving any voltage reading from phase_c to phase_a?