read_mean in addition to read_average

[ClemensGruber]

feature-request

Many thanks Rob for this new HX711 lib! I'm using the library from Bogdan and Andreas since years with your https://github.com/RobTillaart/RunningMedian library. The reason is simple. Outlier have a huge impact for sensor readings and happen in the analog domaine sometimes. So it is a good habit to take the reading several times. The library from Bogdan and yours have a read_average function for this purpose.

Statistically the bad thin on the mean / average funcion is that a hight outlier can kill your average sensor reding in case the outlier is verry high / low. so 568 kg, 25,5 kg, 25,6 kg, 25,3 kg, 25,2 kg will lead to a too high reading also it is averaged.

A more robust statistical value is the mean. No outlier will damage the "real" value and so it is the preferede index for computing an analog input.

So it would be great to have a function like read_mean() to get stabile outputs.

In my bee hive monitorin sketches I have also implemented (not in the lib but in my code) a customizable wait value between the single read_mean / read_average actions. The reason is that in the wild wind can lead to a misreading, so it is good to wait e.g. 1 second between the single read_mean events. That would be also a perfect additional parameter for read_mean and read_average.

Just as an suggestion to make the library more convenient for different scenarios!

[RobTillaart]

Hi Clemens,

If I understand correctly you would like to have a read function that does not calculate the average but does calculate the median of 5 or 11 values. (odd numbers are preferred) . As it sounds not to difficult I will create a new branch so you can do some testing, OK? Might take a few iterations.

I propose to make the following function: float read_median(uint8_t times = 7); It will take 7 reads by default - max 15 to keep memory low .

I will not copy the idea of doing a delay in the function as that would block all code, and while it might be a perfect fit for your application it would not for many others.

So lets start with a non-blocking version.

[RobTillaart]

Please check the branch - https://github.com/RobTillaart/HX711/tree/read_median and verify if it works for you.

fixed 1st bug and a 2nd :)

[RobTillaart]

@ClemensGruber Found time to verify the code ?

[ClemensGruber]

I have tried the read_median branch and added the calibration section from the kitchen scale example to my read median example. I also added Serial.println(scale.get_units(10)); in the main loop to compare different outputs.

From a user perspective I would expect that read_mean returns a tared and calibrated value, e.g. 20,4 kg so I fear my comparison with read_average was a bit missleding because read_average returns raw values and not a unit. read_median is a good starting point but should lead to a similar function as scale.get_units() but not based on read_average() but read_median() function.

I don't know what the benefit would be in case we have two functions scale.get_units() with median and with mean. I see no advantage to use the mean. So I would switch to median only. :-)

In case this is to heavy two functions would be possible or a variable where I can specify if the base for scale.get_units() is read_mean or read_average.

Btw the output is currently

Put a 1 kg in the scale, press a key to continue
UNITS: 1000.46

So I would change the text to "put 1000 g in the scale" so units gram fits to the output 1000.46 g not kg.

This is the output of the original HX_read_median sketch on an ESP32 (TTGO T-Call). The performance test feels quite long with 1 minute for 100 readings but I have not tested with other microcontrollers. So I don't know it this is in an expectable range.

LIBRARY VERSION: 0.2.2

PERFORMANCE
100x read_median(7) = 61805147
  VAL: 3708.00
3681.00
3695.00
3718.00
3702.00
3670.00

Changing f = scale.read_mean(7); to f = scale.read_average(7); leads to nearly the same time output

PERFORMANCE
100x read_average(7) = 61808088
  VAL: 26538.14
3654.00
3651.00
3673.00
3672.00

[RobTillaart]

I like the idea that get_units() could work on both the average or the median.

The change should be in get_value(); something like this?

float get_value(uint8_t times = 1)
{
  if (use_median == true)     
  {
    return read_median(times) - _offset;
  }
  return read_average(times) - _offset;
}

void set_median_mode() { use_median = true;  };
void set_average_mode() { use_median = false; };
bool uses_median() { return use_median; };

It should be in the documentation that use_median will limit the # samples (times) taken due to the internal storage limit.

---

About the time used, I would have expected that read_median(7) would take a bit longer than read_average(7). read_average has an expensive division and median has more additions(subtractions to compare)

A quick test (with no device connected) on AVR also gives approx. equal timing for both, Note the underlying read() takes almost all the time

100x read_median(7) = 282230 == way less than 60 seconds by the way 100x read_average(7) = 282960 700x read() = 278548 (98.4% of time)

---

Changed 1Kg to 1000 gr in the Kitchen example as that is better.

[RobTillaart]

Added the following in read_median branch

  // get set mode for get_value() and indirect get_units().
  void     set_median_mode() { _mode = HX711_MEDIAN_MODE; };
  void     set_average_mode() { _mode = HX711_AVERAGE_MODE; };
  int      get_mode() { return _mode; };

get_value() uses the _mode for selecting read_average() or read_median()

Please verify this works for you.

It might be possible to add new modi operandi in the future e.g.

• MODUS (most frequent occurring value which is difficult as we work with floats)
• AVERAGE_MEDIAN make 15 readings and take the average of the middle 7 after sorting. (find a good name)

The latter is interesting as it might even be better than just the MEDIAN. It removes both outliers and suppresses "small" noise

[ClemensGruber]

About the performance: I have changed the library to the one from Bogdan and Andreas https://github.com/bogde/HX711 testwise and got with 61 seconds nearly the same performance time:

PERFORMANCE
100x read_average(7) = 61807535
  VAL: 3719.00

After that I used this test code (back with this lib here :-)

  scale.set_median_mode();
  Serial.print(scale.get_units(7));
  Serial.print("\t");

  scale.set_average_mode();
  Serial.print(scale.get_units(7));
  Serial.println();

and computed about 500 values each.

The mean from all median readings was comparable with the mean readings (median: 1003.73 vs. mean: 1003.69), SD was a bit samller for median 2.34 vs. 2.80 as expected, minimum / maximum was 965.8 / 1008.0 vs. 965.7 / 1011.8, all in gram!

Not too much differenc but it could also be up to my setup: workbench not outside, short cables and especially a 100 kg load cell loaded with 1 kg only

[RobTillaart]

@ClemensGruber Thanks for your feedback, very valuable.

Can you do a test in which you add another 1 Kg weight in the last 15 seconds to simulate an outlier. The median mode should return the original weight, while the average would be affected and return a higher weight. In fact given that the time of the second weight is known you could estimate how must the average would be affected.

(15 seconds is ~25% of 1 minute so expectation is 75% 1Kg + 25% 2Kg ==> 1.25 Kg on average.

[RobTillaart]

Started to implement the MEDAVG methode. (if you know a better name, let me know)

the idea is given in this sketch:

//    FILE: medium_average.ino
//  AUTHOR: Rob Tillaart
//    DATE: 2021-05-13

void setup()
{
  Serial.begin(115200);

  for (int times = 3; times < 20; times++)
  {
    uint8_t first = (times + 2) / 4;
    uint8_t last  = times - first - 1;

    Serial.print(times);
    Serial.print("\t");

    for (int i = 0; i < first; i++) Serial.print("0");
    for (int i = first; i <= last; i++) Serial.print("1");
    for (int i = last + 1; i < times; i++) Serial.print("0");
    Serial.println();
  }

  Serial.println("done...");
}

void loop(){}

output:

3   010
4   0110
5   01110
6   001100
7   0011100
8   00111100
9   001111100
10  0001111000
11  00011111000
12  000111111000
13  0001111111000
14  00001111110000
15  000011111110000
16  0000111111110000
17  00001111111110000
18  000001111111100000
19  0000011111111100000
done...

sort the readings, take the average of the elements that have a 1 as the 0's are the outliers.

[RobTillaart]

@ClemensGruber new version pushed in read_median branch

Please verify it is working for you,

[RobTillaart]

@ClemensGruber any progress testing?

[RobTillaart]

@ClemensGruber I merged the read_median branch into master. If you find any bugs or need a new feature, please open a new issue,

Thanks

[ClemensGruber]

Many thanks, Rob for merging this feature into master. So it is easily available via Arduino's library manager! I installed the lib now via the library manager and tested scale.set_median_mode(); vs. scale.set_average_mode();

//  scale.set_median_mode();
  scale.set_average_mode();

  Serial.println("Start reading ...");
  Serial.println(scale.get_units(15));

I used the maximum of 15 readings by calculating the value and gave the scale a short press while reading. The results where clearly different. I had nearly same values with the median mode and wide variability with the average. So all seems to work in a good manner!

The performance is - also as expected - quite similar for the different modes on my ESP32:

0   PERFORMANCE set_average_mode
100x set_average_mode = 61806866

1   PERFORMANCE set_median_mode
100x set_median_mode = 61807794

2   PERFORMANCE set_medavg_mode
100x set_medavg_mode = 61806749

Again many thanks for implementing this!

[ClemensGruber]

Changed 1Kg to 1000 gr in the Kitchen example as that is better.

There is a second occurrence in the grocery scale example that you may like to change also:

https://github.com/RobTillaart/HX711/blob/cc18abc6d9d52ecf9179ba502be71aa9520926c1/examples/HX_grocery_scale/HX_grocery_scale.ino#L42

[RobTillaart]

@ClemensGruber Will be fixed in 0.2.3 when I add running average to the list of modi operandi.