Inaccurate Flow measurement

[booboodoguy]

I tested the flow rate measurement capability of OSPI 2.2.0 on a raspberry pi zero 2w. I attached a pulse generator to pin 8 (GPIO14) and set the pulse rate to 1. Per Rays advice stating, 10 clicks per second max pulse rate, I set the rate to 4hz 50% duty cycle ( 4 pulses per second). I manually ran a zone for 15 minutes. The reported flow rate on the UI during the on time ranged from 190 to 224. Each UI update was different.

I next reduced the pulse rate down to 0.5hz ( 1 pulse per 2 seconds). The reported flow rate on the UI ranged from 26 to 78. Approximately every other UI update was different. The magnitude of the flow rate variance limits the usefulness of this measurement capability. Is there a way to improve the measurement accuracy ?

[rayshobby]

I've personally verified the flow sensing many times before on RPi and as far as my experiments went, it was quite accurate. I don't have an RPi zero 2w so I cannot verify it for this particular version of RPi, but I've verified it on RPi 0 w, Rpi 2, Rpi 3 etc. What type of pulse generator are you using? Do you have a link? Also, what's your version of raspbian?

[booboodoguy]

Buster Lite, frequency generator is accurate to < 5ppm. Flow signal was verified by a scope on the header flow pin. My testing has concluded that the flow sense measurement is distorted if there is a wifi UI connection active during a running program. When no wifi connection is active the flow measure is accurate.

[rayshobby]

"What type of pulse generator are you using? Do you have a link?"

[booboodoguy]

UDB1000S 8MHZ DDS Function signal generator UDB1000_EN_manual.pdf

https://www.aliexpress.us/item/3256801384491976.html?spm=a2g0o.detail.pcDetailTopMoreOtherSeller.3.27c7BdwIBdwIXn&gps-id=pcDetailTopMoreOtherSeller&scm=1007.40050.354490.0&scm_id=1007.40050.354490.0&scm-url=1007.40050.354490.0&pvid=af312d37-87db-4299-8e4d-9edebf2bb88b&_t=gps-id:pcDetailTopMoreOtherSeller,scm-url:1007.40050.354490.0,pvid:af312d37-87db-4299-8e4d-9edebf2bb88b,tpp_buckets:668%232846%238110%231995&pdp_npi=4%40dis%21USD%2170.00%2132.90%21%21%2170.00%2132.90%21%402103205117093988398971616eeadb%2112000016631819887%21rec%21US%21163386137%21&utparam-url=scene%3ApcDetailTopMoreOtherSeller%7Cquery_from%3A

[rayshobby]

Are you aware that the sensors should be dry contact switch type sensors (e.g. rain sensor, reed switch sensor)? If you use a signal generator, it should have open-drain output and should not output an active voltage. Since you are using a function generator, it clearly outputs an active voltage. If you do so, you have to make sure the output voltage is a square wave that swings between 0V to 3.3V (it must not swing above 3.3V and cannot be negative). I am not sure if you can set your function generator to swing between 0 to 3.3V, most of them by default will swing from -Vcc to +Vcc. You will risk damaging your RPi if it swing negative. These sort of issues do not exist if you use a dry contact switch as dry contact switch does not insert an active voltage.

[booboodoguy]

Yes, I'm aware of all your good advice. I set the freq generator output to 3v. I have also run the tests using an optical isolator circuit to drive the header pin. The opto output is pulled up to the 3.3v power through a 10k resistor. This is what the actual flow sensor output is connected to. Connecting a flow detector output directly to the controller which is outdoors under ground and many feet away from the controller would not be a wise thing to do. Even with diode clamping to the power rails for protection is risky.

Can you run a test using the latest firmware as I have?

[booboodoguy]

Update on flow measurement accuracy. As it turns out the do_loop in main.cpp is "calling" EthernetClient EthernetServer::available() in etherport.cpp every loop cycle. EthernetClient EthernetServer::available() inserts a 50ms delay of dead time and when added to the rest of the do loop cycle time results in the flow_poll missing pulses when the flow signal frequency is approximately greater than 8hz.

My flow sensor output ranges from 7 to 15 HZ so I reduced the 50ms to 20ms and now flow_poll is very accurate.

It's not clear to me why EthernetClient EthernetServer::available() has to be "called" every time through the do loop or why a 50ms delay is required when using "modern" wifi communication.

[rayshobby]

The etherport.cpp for Raspberry Pi was adopted 9 years ago: https://github.com/OpenSprinkler/OpenSprinkler-Firmware/commit/23f18abc62aa5dec6b0c5dc05732a91c4e790fd1 From the commit message it looks like the class came from rzsimm's sprinklers_pi project. From your description it does sound like this would cause accuracy issues for flow sensor. I think this issue didn't rise up sooner due to a couple of factors: 1) the professional flow meters that we recommend usually send clicks no more than 1Hz, so it's largely not affected by the 50ms poll delay; 2) before the more recent mess with RPi's GPIO controls (they changed the way GPIOs work several times), the flow sensor used to be handled by interrupt functions, which are not affected by delays in the main loop. We will investigate this issue soon and either shorten the delay or adopt a better, non-blocking method.

To answer your other question: EthernetServer::available() should be called every loop iteration because the server needs to respond to web requests in a timely matter. This part of the code is the same across AVR/ESP/RPi/Linux. The main issue is in the available() function as you pointed out, which should be either non-blocking or use a very short blocking delay.

[booboodoguy]

Thank you for the response. Given the EthernetServer::available() delay (even if it is reduced), I suppose the flowpoll timeout is not necessary in the do_loop as well as the 1ms delay code.

// handle flow sensor using polling every 1ms (maximum freq 1/(2*1ms)=500Hz)
static ulong flowpoll_timeout=0;
if(os.iopts[IOPT_SENSOR1_TYPE]==SENSOR_TYPE_FLOW) {
    ulong curr = millis();
    if(curr!=flowpoll_timeout) {
        flowpoll_timeout = curr;
        flow_poll();

#if !defined(ARDUINO)
    delay(1); // For OSPI/OSBO/LINUX, sleep 1 ms to minimize CPU usage
#endif

}

[rayshobby]

First, in the future could you just post links to the line of code, instead of pasting the code directly? This makes it easy to find exactly where in the code you are referring to. For example, the flow_poll and delay(1) are in two separate locations, the way you pasted the code made it look like they are in the same block, but they aren't.

Second, the two 1ms delays are not the same: for flow_poll, it's there to make sure we aren't calling flow_poll all the time, because most flow sensors aren't more than 500Hz. It's also there to serve as a 'software' low-pass filter. Without it, a very noisy sensor can result in a lot of false triggers. The delay(1) at the end of the main loop is for reducing CPU usage. As far as I can remember, without delay(1) the firmware will basically throttle the CPU making it unable to run other programs/processes. It may not be necessary now with the latest Raspbian but it needs to be tested.

I would much prefer improving etherport.cpp and use non-blocking method to implement available() than to change the way flow_poll is executed. It's partly because this is I believe the correct way to fix the issue, partly because the same firmware code runs on all OpenSprinkler platforms, so I would rather fix the issue in RPi/Linux specific branch than to change the code that affects all other platforms.

[booboodoguy]

I agree a non blocking solution would be the preferred way to handle this. BTW thank you for all your support.

[rayshobby]

Sure thing. Thanks for finding out the source of the issue. I definitely didn't think it could be in the etherport class.

[rayshobby]

This has been addressed in PR #288 which will be released as firmware 2.2.1(0).