pulse_counter report total pulses with dynamic interval

[FrankBakkerNl]

Describe the problem you have/What new integration you would like

I would like the pulse counter to also report the total number of pulses instead of just the current pulses / minute. Even nicer would be if the pulse count would be persisted in the eprom, so that it is kept over reboots.

To avoid wearing out the eprom, the persistence interval could be made configurable, something like: Persist every x pulses, but no more than once every n seconds and at east once every m seconds

Please describe your use case for this integration and alternatives you've tried:

I use the pulse counter to monitor energy consumption from a Kwh meter with an S0 port. I would like to keep track of total Kwh's used. By storing the pulses en reporting total pulse count has some advantages:

• If somehow messages do not arrive at the home assistant I still have a complete total of energy consumption
• Higher accuracy of total consumption as it is directly derived form total pulse count
• Configuration on the home assistant becomes easier as it does not have to do integral calculations over time

Additional context

[FrankBakkerNl]

After some more investigation I found that the total_daily_energy sensor partly does what I described. Especially in combination with a filter.

I think however the 'daily' reset is very specific. It would be nice to have a configurable reset interval, or no reset at al. The name implies it is specific for energy counters, but I would assume any pulse based counter can be used.

Still it seems a bit weird to first translate a counter to a #/sec and then back to a total. This would still argue for adding a total count into the pulse counter itself.

[expaso]

I was thinking about the pulse counter myself, as Im also using it on an s0 meter and a water meter.

There are 2 things we want to measure here:

1- The active power in Watts, or the current waterflow in l/min 2- The usage over time. In kwh or m3

For number 1 the pulse counter is almost unusable. You want accurate data of the power/water used, and want the feedback to be quick, within seconds. The pulse counter cannot satisfy that, because it simply counts pulses per min. Shortening this makes it less accurate. It should measure the time between 2 pulses instead of counting them for this one.

For number 2, the pulse counter together with the integration sensor, or daily energy sensor, it's highly accurate. You know that with every pulse, a fixed unit is consumed. So counting pulses is counting units.

The daily energy sensor does just that. It sums per day, and then resets. It resets per day, because you KNOW that one day your esp resets due to reboot/update/power surge. Keeping this counter running forever would lead to this counter getting massively out of sync with the real powercounter anyway. In dashboarding, you probably want to show the used energy per day/week/month, and not the current counter value anyway.

So energy per day is a tradeoff. You WILL miss pulses from those counters due to ESP reboots and such, so you WILL get out of sync eventually. Wearing out your flash by storing counters wont help with these missed pulses. Cumulative Daily energy will also reset due to above conditions, but these resets can be easily detected in dashboarding queries which will just sum the peaks per day.

That said, if enough interest I will try a PR to fix nr 1. For NR 2.. Hmm.. Maybe EEPROM rotation could help to store a cumulative total somehow. Anybody interested in this?

[FrankBakkerNl]

My first 'kind of working' solution was a pulse_counter with an integration sensor. However I got some inf values which, when adding up result in inf, not sure maybe something in my configuration that caused a devision by zero. The root problem is that the Pulse_counter does a differential calculation which then needs to be integrated again. This undoubly leads to rounding errors which I think are far worse than missed pulses during reboots.

In other words, kWh should be calculated directly using the pulsecount x factor and not by integrating watts over time. Watts can the be calculated by differentiation of wkh or pulses with some factor.

My new solution I just installed now has a Custom_component that is a Sensor that just counts all pulses and reports that value. I then use a template sensor to calculate kWh (just multiply by a constant factor. Another, bit more complicated template sensor does a differential calculation to get to watts. I will post my code shortly (not nearly PR ready 🤫)

Because my kwh meter is monitoring the charging of my car it is idle most of the time. Chances that I will miss a lot of pulses during reboot are small, but I agree cannot be removed totally. Therefore i did implement storing the raw value with some smart logic, it shouls take at least 2 years to wear or the flash, which I think is reasonable.

[FrankBakkerNl]

#include "esphome.h"
#include <arduino.h>

volatile long pulseCount = 0;
const int pin = 5; // D1

// save every # of pulses
const int saveMaxPulseInterval = 80; // 0.1 kwh because 800 pulses / kwh

// minimun time between saves
const int saveMinTimeInterval = 60 * 1000; // 30 seconds

// save at least once every
const int saveMaxTimeInterval = 10 * 60 * 1000; // 10 minutes

// The interrupt routine
void ICACHE_RAM_ATTR onPulse()
{
  pulseCount++;
}

class Pulse_Count_Total : public PollingComponent, public Sensor
{

private:
  //  Store values between updates
  long lastSavedValue = 0;
  uint lastSavedTime = 0;

  ESPPreferenceObject rtc_;

public:
  Pulse_Count_Total() : PollingComponent(1000) {}

  void setup() override
  {
    this->rtc_ = global_preferences.make_preference<long>(this->get_object_id_hash());

    long currentValue = 0;
    this->rtc_.load(&currentValue);
    ESP_LOGD("Pulse_Count_Total", "Pulse value loaded from preferences is: %i", currentValue);

    // uncomment to reset value
    // currentValue = 0;
    // this->rtc_.save(&currentValue);

    pulseCount = currentValue;

    // initialize Save state
    lastSavedValue = currentValue;
    lastSavedTime = millis();

    pinMode(pin, INPUT_PULLUP);
    attachInterrupt(digitalPinToInterrupt(pin), onPulse, RISING);
  }

  void update() override
  {
    // copy value just to make sure it is not changed between publish and save
    long value = pulseCount;
    publish_state(value);
    SaveIfNeeded(value);
  }

  void SaveIfNeeded(long value)
  {
    uint currMills = millis();
    uint pulseDelta = pulseCount - lastSavedValue;
    uint timeDelta = currMills - lastSavedTime;

    if ( // save after 1 pulse if saveMaxTimeDelta  has passed
        (pulseDelta > 0 && timeDelta > saveMaxTimeInterval) ||

        // save after saveMaxPulseInterval pulses if saveMinTimeInterval has passed)
        (pulseDelta > saveMaxPulseInterval && timeDelta > saveMinTimeInterval))
    {

      long value = pulseCount;
      lastSavedValue = value;
      lastSavedTime = currMills;
      this->rtc_.save(&value);
    }
  }
};
`

[FrankBakkerNl]

sensor:
  - platform: custom
    lambda: |-
      auto my_sensor = new Pulse_Count_Total();
      App.register_component(my_sensor);
      return {my_sensor};

    sensors:
      name: "RawPulsesS0"
      id: "RawPulsesS0"
      internal: true

  - platform: template
    name: "Total consumption EV charger "
    unit_of_measurement: 'kWh'
    accuracy_decimals: 4
    lambda: |-
      const float pulsesPerKwh = 800;
      return id(RawPulsesS0).state / pulsesPerKwh;
    update_interval: 10s
    filters:
    - throttle: 10s
    - or:
       - throttle: 5min
       - delta: 0.1

  - platform: template
    name: "Current power EV Charger"
    unit_of_measurement: 'W'
    accuracy_decimals: 8
    lambda: |-
      uint currTime = millis();

      // init statics once
      static uint lastPulseCount = 0;
      static uint lastTime = currTime;

      uint elapsedMs = currTime - lastTime;
      // check for zero timespan
      if (elapsedMs == 0) return 0;

      uint currPulseCount = id(RawPulsesS0).state;
      float pulseDelta = currPulseCount - lastPulseCount;

      const uint pulsesPerKwh = 800;
      const uint msPerHour = 60 * 60 * 1000;

      float wh = pulseDelta * 1000 / pulsesPerKwh;
      float watt = wh * msPerHour / elapsedMs;

      // save current values for next time
      lastTime = currTime;
      lastPulseCount = currPulseCount;

      return watt;
    update_interval: 10s

`

[FrankBakkerNl]

Sorry for the long code snippets here (with poor formatting) , but I did want to share what I did and this seemed the easiest way to do it

[FrankBakkerNl]

As a sidenote, I tried to do the configuration of my custom_component, like the pin number etc, in the yaml, like the standard sensor do, but I was unable to get that working, any ideas on how to do that?

[FrankBakkerNl]

My suggestion for a flexible solution is:

Make pulse_counter (or a new sensor) report the absolute number of pulses.

Add an option to save the absolute value with some interval.

Create a flexible helper sensor to differentiate over a configured time period (what I did in my template sensor but then more generic with less code in the yaml). This way a user can choose to report pulses per second, hour, day whatever.

[expaso]

@FrankBakkerNl Looks like you are nearly there already ;)

Can we not make it even more userfriendly and create a power and energy sensor on top of that?

• A power-sensor for the current Watts, which uses time between 2 pulses and resets on a given timeout. (which won't be completely accurate, I know)
• An Energy-sensor to accumulate the pulses and outputs the total Kwh / m3 / whatever, storing the intermediate values in flash using EEPROM rotation and a given time-interval?

[rspaargaren]

My suggestion for a flexible solution is:

Make pulse_counter (or a new sensor) report the absolute number of pulses.

Add an option to save the absolute value with some interval.

Create a flexible helper sensor to differentiate over a configured time period (what I did in my template sensor but then more generic with less code in the yaml). This way a user can choose to report pulses per second, hour, day whatever.

I am having a short look on creating a pulse counter reporting the absolute values. This is not a big deal and try to make a branch. Internally the pulse counter is already counting the total number of pulses...