search

EnviroDIY / ModularSensors

An Arduino library to give environmental sensors a common interface of functions for use with Arduino-framework dataloggers, such as the EnviroDIY Mayfly.
https://envirodiy.github.io/ModularSensors/
Other
81 stars 48 forks source link

Develop wrapper for Meter Teros 11 Soil Moisture Sensors #276

Closed aufdenkampe closed 1 year ago

aufdenkampe commented 5 years ago

The Meter Group purchased Decagon about a year ago, and just recently stopped selling the 5TM soil moisture sensor, replacing it with the Teros 11 Soil Moisture + Temperature sensor, which uses a more robust construction design that Decagon had been piloting.

Unfortunately @Volk3rJ discovered that the SDI-12 commands are no longer the same. These differences include:

  1. A change in the sensor response, where the Teros 11 gives raw counts rather than EA values as 1st of 2 returned variables. See https://github.com/LimnoTech/EnviroDIY_deployments/commit/23a6f688ae6edadf3a5e91030296bd79364fc9c4
  2. A change in the SDI-12 timing, which gives lots of errors when more than one Teros 11 sensor is attached to the same Mayfly.

@Volk3rJ partially solved #2 by placing the sensor-variables as far apart as possible in the variables array. He also noticed some hints of these differences in the TEROS 11/12 INTEGRATOR GUIDE and the TEROS 11/12 USER MANUAL.

Volk3rJ commented 5 years ago

Regarding the SDI-12 timing issue:

The sensor that gets called first responds normal, the sensor called second only reads error values (or maybe doesn't respond at all?). I partially fixed this by calling modem RSSI and other values in the array between 1. and 2. sensor. This curiously works for one of my stations but not for another!

I also tried to increase the sensor delays and timings in the 5TM.h file without any luck

aufdenkampe commented 5 years ago

I probably need to add documentation to Wiki to fully close.

aufdenkampe commented 5 years ago

@Volk3rJ, as you might have noticed above, I have this working with two Teros 11 sensors. I made a couple of timing tweaks relative to what you did, so I'm not sure what solved it.

Also, my station with two Teros 11 sensors does not also have a MaxSonar (but it does have a BME280), so that is also worth testing.

aufdenkampe commented 3 years ago

Note that the timing issues here popped up again for @shepardb when he updated the ModularSensors library on a station deployed with 2 Teros 11 sensors that had been working fine for a year.

aufdenkampe commented 3 years ago

Reopening and looping @GArrigotti-cws into this issue, so we can fix this issue that has returned since Modular Sensors versions >0.25ish (@shepardb noticed it summer of 2020).

SRGDamia1 commented 3 years ago

The changes I just made with v0.28.4 might help this. I made the time between the SDI-12 break and the actual command settable per sensor. I discovered with the newest Meter sensor we got, it was not responding properly with even the 10ms delay I'd added for the RDO. I set the delay for the Teros (and 5TM) back to 0, which is what it was back in 2019. If it doesn't fix the problem, it at least gives one other aspect of the timing that you can try to tweak to see what works.

For any sensor that reports back a manufacturer of "METER" I also added code to turn off the DDI serial output string. If the sensor isn't at the factory default address of 0, it shouldn't be putting out the string anyway, but it seem safer to turn it off just in case it was causing extra garbage on the SDI-12 line.

GArrigotti-cws commented 3 years ago

@SRGDamia1 I did tweak a couple of those and still had some issues, I'll see if that change in modular sensors partially rectifies.

SRGDamia1 commented 3 years ago

@GArrigotti-cws - it's a bit of a long-shot, but maybe it changes something.

In the 'tools' section of the SDI12 library there are a few scripts that I made to test sensor timing - basically repeating everything over and over with slight changes to the warm-up, wake delay, etc. They might help.

GArrigotti-cws commented 3 years ago

@SRGDamia1 Okay, I'll give it a try. Thank you.

aufdenkampe commented 3 years ago

@SRGDamia1, thank you so much for describing these issues and making the new release!

As @GArrigotti-cws, mentioned, a couple of weeks ago the two of us modified the Teros 11 timing settings in src/sensors/MeterTeros11.h:

/// @brief Sensor::_warmUpTime_ms; the Teros 11 warm-up time in SDI-12 mode:
/// 245ms typical
#define TEROS11_WARM_UP_TIME_MS 250
/// @brief Sensor::_stabilizationTime_ms; the Teros 11 is stable after 50ms.
#define TEROS11_STABILIZATION_TIME_MS 50
/// @brief Sensor::_measurementTime_ms; the Teros 11 takes25 ms to 50 ms to
/// complete a measurement.
#define TEROS11_MEASUREMENT_TIME_MS 50

I think we increased the warm time to seconds, but it didn't make a difference.

If I understand correctly, with this commit https://github.com/EnviroDIY/ModularSensors/commit/a6eee7595b4b3cbe55df761818ac3759dda5d137#diff-c2718078e9b993d65131fd0cc535a573e7e2d2130591c9a77021d3f7519ae383 \ you added this:

/// @brief Extra wake time required for an SDI-12 sensor between the "break"
/// and the time the command is sent.  The Terros-11 requires no extra time.
#define TEROS11_EXTRA_WAKE_TIME_MS 0

and that we might want to try increasing it from 0 to 10 ms or maybe higher. Is that right?

aufdenkampe commented 3 years ago

@GArrigotti-cws, the other thing we'll try is to power the Teros 11 with 5V. I just learned from @SRGDamia1 that the Mayfly's ATmega1284p microprocessor can actually handle up to 6V.

To protect the Mayfly, we've always powered Meter sensors at 3.3V to match the Mayfly's voltage. Although most Meter sensors say they need a minimum power of 3.6 V to 4.0 V, we have found that they operate fine at 3.3. However, maybe the Teros 11 is more sensitive at low voltage. Also, it's stated minimum is 4.0 V, unlike many of the others that list 3.6V.

See http://publications.metergroup.com/Integrator%20Guide/18224%20TEROS%2011-12%20Integrator%20Guide.pdf

SRGDamia1 commented 3 years ago

Yes, you could try increasing the extra wake time. For a long time it was zero, then I bumped it up to 10ms for the RDO. I found the newest Meter sensors did not respond consistently with the delay so I made it an argument so it could be set back to 0 for the Meter sensors.

The extra wire time applies to every SDI12 command sent to the device. Since the whole SDI12 is designed for environmental sensors; the sensors are required to go to sleep any time except when they are being addressed or measuring. The sensors are required to be ready to listen for a new command within 100 ms of a "break" and "marking" on the SDI12 line. The Meter sensors don't need any of that 100ms, they're ready within the space of the break and marking. The newest CTD we got actually was a little flaky when given the extra time. I think some of the Campbell sensors need the full 100ms.

So, for a Meter sensor, I suspect the right amount of time is 0, but you can try increasing it. It should never be over 100.

Also, the maximum voltage on a pin for the 1284p is dependent on the voltage it's powered at; it should be no more than 0.5V higher than the power voltage. The Mayfly powers it with 3.3V, so the pin max is 3.8V. It's as much voltage differential as absolute voltage. The very maximum power voltage it can handle is 6V, but that's not what the Mayfly has. So, while I really don't think short pulses like SDI12 at over voltage are really likely to be problematic, I definitely would not recommend regularly subjecting the Mayfly pins to 5V for long periods of time.

aufdenkampe commented 3 years ago

@SRGDamia1, that's a very helpful explanation! Thanks.

I just looked again at the Meter Teros 11-12 Integrators Guide, and it says that output voltage is typically 3.6V!

I had thought the SDI-12 specification was 5V for SDI-12 sensors, and presumed the Teros 11 followed that.

So for the Meter Teros 11 or 12 sensors, even if they are powered at ~12V, their digital high signals should still be ~3.6V.

The Meter Hydros 21/CTD Integrators Guide shows the digital output voltage of ~ 3.6V.

So, it seems we should be powering these Meter sensors at 5V.

neilh10 commented 3 years ago

(EDIT oops just reading on and Sara said the same thing) Hello I'm wondering where the info on the mega1284 handling up to 6V (on a input pin from SDI-12?) came from. I read from the data sheet for the Mayfly it would be a max of 3.8V?

In the data sheet I've been using from the BOM ATMEGA1284P_VQFN44 or ATMEGA1284P-MU it says the Absolute maximum ratings, Voltage on any pin except RESET is to Vcc+0.5V. Mayfly Vcc is 3.3V, so that would mean a max of 3.8V on any input pin. It does say that the Maximum operating voltage is 6.0V, Vcc can be 1.8Vto5.5V, and DC Current per I/O pin 40mA - though test conditions for 3V is a sink of 10mA

image https://ww1.microchip.com/downloads/en/DeviceDoc/ATmega164A_PA-324A_PA-644A_PA-1284_P_Data-Sheet-40002070B.pdf

neilh10 commented 3 years ago

Reading the Meter Teros 11 spec, it does fit with what SDI-12 spec.
I would say for reliability and repeatability you want to power it with 4V or more per the Meter Teros 11 spec. It may work at a lower voltage in the lab, but might not across all operating voltages of -40C to +60C. The SDI-12specs says the data-line should exceed 3.5V ... but may be up to 5.5V - that fits with the Teros 11 Typical 3.6V. Teros 11 would still be in spec to supply 5.5V (maybe this only happens on a blue-moon at -10C, but would still be in specification - but might cause problems for the Mayfly ) or maybe it never happens due to circuit design. My understanding of crimping the specifications, is that you may get something working in nice conditions indoors, but what happens in the great outdoors. My tradeoff is to limit the current into the Mayfly with a series R ~ not totally correct, but seems like its closer to the requirements of what is needed. That is also the SDI-12 recommended surge protection image

aufdenkampe commented 3 years ago

@neilh10, just be clear, I had misunderstood the comment from @SRGDamia1 about the Mayfly's 1284p being able to handle 6V. She clarified in https://github.com/EnviroDIY/ModularSensors/issues/276#issuecomment-833143534 that the 1284p is only specified for that voltage if it is also powered near that voltage, and that the plus/minus 0.5V is still the spec.

Thanks also for noticing that extra info about SDI-12 voltage ranges.

It's clear that I should try powering these sensors with 5V.

aufdenkampe commented 3 years ago

@GArrigotti-cws and I are testing right now. Seems like 5V help a bit. We now have both sensors working in Sensor Testing Mode, but only one works in regular logging mode.

aufdenkampe commented 3 years ago

Update from testing this afternoon:

Here's the example of it working with both debug statements defined:

> Executing task: C:\Users\ArrigottiG\.platformio\penv\Scripts\platformio.exe device monitor <

--- Available filters and text transformations: colorize, debug, default, direct, hexlify, log2file, nocontrol, printable, send_on_enter, time
--- More details at http://bit.ly/pio-monitor-filters
--- Miniterm on COM4  115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---

Now running Arrigotti_NoRadio.ino on Logger Arrigotti

Using ModularSensors Library version 0.28.4
Logger timezone is set to UTC -8
RTC timezone is set to UTC
Current RTC time is: 2000-05-19T07:09:32-08:00
All variable UUID's appear to be correctly formed.

2a5f077b-8de9-45ee-9704-4743726101fb -> SoilEa
168081de-2c1a-458f-8f86-1d88f7b8c394 -> SoilTemp
d7a3e8c8-d1f0-4503-a76f-93f5b6861850 -> SoilVWC
e5d304df-995b-4272-9a26-f3b525c315b7 -> BoschBME280Temp       
f8984e98-bbd4-4c29-a178-a64bcd546411 -> BoschBME280Humidity   
6872a270-5dfb-4cee-9326-118adc97af9e -> BoschBME280Pressure   
c3a10820-59ef-48a0-9fa5-88bb06914186 -> Battery
61e38606-b91c-4058-ac8c-3ff1ff280ede -> BoardTemp
29abed47-770f-44ef-893e-bff2645a1ff9 -> SoilEa
aa142833-d52e-4f91-b2b7-d3fa953acdae -> SoilTemp
49d31d59-1986-4a9c-ab6f-16d7639c8abd -> SoilVWC

There are 0 calculated variables in the group. <--VariableArray
There are 0 calculated variables in the group. <--VariableArray
This logger has a variable array with 11 variables, of which 11 come from 5 sensors and 0 are calculated.
Logger portion of setup finished.

Setting up sensors...
Beginning setup for sensors and variables... <--VariableArray
Running sensor setup functions. <--VariableArray
    EnviroDIY Mayfly at EnviroDIY Mayfly was already set up! <--VariableArray
    Set up of MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
Enabling interrupts for SDI12 on pin 7 <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors   
    MeterTeros11 at SDI12-1_Pin7 did not reply! <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Getting sensor info <--SDI12Sensors
   SDI12 Address: 1 <--SDI12Sensors   
   SDI12 Version: 1.30 <--SDI12Sensors
   Sensor Vendor: METER <--SDI12Sensors
   Sensor Model: TER11 <--SDI12Sensors
    Sensor Version: 112 <--SDI12Sensors
   Sensor Serial Number: T11-00006560 <--SDI12Sensors
  Suppressing DDI string on Meter sensor <--SDI12Sensors
        ... setup succeeded. <--VariableArray
    Set up of BoschBME280 at I2C_0x76 ... <--VariableArray
        ... setup succeeded. <--VariableArray
    Set up of MaximDS3231 at I2C_0x68 ... <--VariableArray
        ... setup succeeded. <--VariableArray
    Set up of MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
Enabling interrupts for SDI12 on pin 7 <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Getting sensor info <--SDI12Sensors
   SDI12 Address: 6 <--SDI12Sensors
   SDI12 Version: 1.30 <--SDI12Sensors
   Sensor Vendor: METER <--SDI12Sensors
   Sensor Model: TER11 <--SDI12Sensors
    Sensor Version: 112 <--SDI12Sensors
   Sensor Serial Number: T11-00006574 <--SDI12Sensors
  Suppressing DDI string on Meter sensor <--SDI12Sensors
        ... setup succeeded. <--VariableArray
... Success! <--VariableArray
Setting up file on SD card
Data will be saved as Arrigotti_2000-05-19.csv
Putting processor to sleep

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING -----

The current clock timestamp is not valid!
----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING -----
------------------------------------------
Creating a mask array with the uniqueness for each sensor.. <--VariableArray
Creating an array for the number of completed measurements.. <--VariableArray
Creating an array with the number of measurements to average.. <--VariableArray
Creating an array of the power pins.. <--VariableArray
Creating arrays of the power pin locations.. <--VariableArray
----->> Clearing all results arrays before taking new measurements. ... <--VariableArray
   ... Complete. <<----- <--VariableArray
----->> Powering up all sensors together. ... <--VariableArray
Powering up sensors... <--VariableArray
    Powering up MeterTeros11 at SDI12-1_Pin7 <--VariableArray
    Powering up BoschBME280 at I2C_0x76 <--VariableArray
    Powering up EnviroDIY Mayfly at EnviroDIY Mayfly <--VariableArray
    Powering up MaximDS3231 at I2C_0x68 <--VariableArray
    Powering up MeterTeros11 at SDI12-6_Pin7 <--VariableArray
   ... Complete. <<----- <--VariableArray
6 . 1 --->> Starting reading 1 on EnviroDIY Mayfly at EnviroDIY Mayfly ... <--VariableArray
   ... set up succeeded. <<--- 6 . 1 <--VariableArray
6 . 1 --->> Collected result of reading 1 from EnviroDIY Mayfly at EnviroDIY Mayfly ... <--VariableArray
   ... got measurement result. <<--- 6 . 1 <--VariableArray
6 --->> Finished all measurements from EnviroDIY Mayfly at EnviroDIY Mayfly , putting it to sleep. ... <--VariableArray
  ... succeeded in putting sensor to sleep. <<--- 6 <--VariableArray
*****--- 1 sensors now complete ---***** <--VariableArray
7 --->> Waking MaximDS3231 at I2C_0x68 ... <--VariableArray
   ... wake up uccess. <<--- 7 <--VariableArray
7 . 1 --->> Starting reading 1 on MaximDS3231 at I2C_0x68 ... <--VariableArray
   ... set up succeeded. <<--- 7 . 1 <--VariableArray
7 . 1 --->> Collected result of reading 1 from MaximDS3231 at I2C_0x68 ... <--VariableArray
   ... got measurement result. <<--- 7 . 1 <--VariableArray
7 --->> Finished all measurements from MaximDS3231 at I2C_0x68 , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 7 <--VariableArray
6 --->> EnviroDIY Mayfly at EnviroDIY Mayfly powered down. <<--- 6 <--VariableArray
7 --->> MaximDS3231 at I2C_0x68 powered down. <<--- 7 <--VariableArray
*****--- 2 sensors now complete ---***** <--VariableArray
5 --->> Waking BoschBME280 at I2C_0x76 ... <--VariableArray
   ... wake up uccess. <<--- 5 <--VariableArray
5 . 1 --->> Starting reading 1 on BoschBME280 at I2C_0x76 ... <--VariableArray
   ... set up succeeded. <<--- 5 . 1 <--VariableArray
5 . 1 --->> Collected result of reading 1 from BoschBME280 at I2C_0x76 ... <--VariableArray
   ... got measurement result. <<--- 5 . 1 <--VariableArray
5 . 2 --->> Starting reading 2 on BoschBME280 at I2C_0x76 ... <--VariableArray
   ... set up succeeded. <<--- 5 . 2 <--VariableArray
5 . 2 --->> Collected result of reading 2 from BoschBME280 at I2C_0x76 ... <--VariableArray
   ... got measurement result. <<--- 5 . 2 <--VariableArray
5 . 3 --->> Starting reading 3 on BoschBME280 at I2C_0x76 ... <--VariableArray
   ... set up succeeded. <<--- 5 . 3 <--VariableArray
5 . 3 --->> Collected result of reading 3 from BoschBME280 at I2C_0x76 ... <--VariableArray
   ... got measurement result. <<--- 5 . 3 <--VariableArray
5 --->> Finished all measurements from BoschBME280 at I2C_0x76 , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 5 <--VariableArray
*****--- 3 sensors now complete ---***** <--VariableArray
2 --->> Waking MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
   ... wake up uccess. <<--- 2 <--VariableArray
10 --->> Waking MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
   ... wake up uccess. <<--- 10 <--VariableArray
2 . 1 --->> Starting reading 1 on MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
   ... set up succeeded. <<--- 2 . 1 <--VariableArray
10 . 1 --->> Starting reading 1 on MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
   ... set up succeeded. <<--- 10 . 1 <--VariableArray
2 . 1 --->> Collected result of reading 1 from MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
   ... got measurement result. <<--- 2 . 1 <--VariableArray
10 . 1 --->> Collected result of reading 1 from MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
   ... got measurement result. <<--- 10 . 1 <--VariableArray
2 . 2 --->> Starting reading 2 on MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
   ... set up succeeded. <<--- 2 . 2 <--VariableArray
10 . 2 --->> Starting reading 2 on MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
   ... set up succeeded. <<--- 10 . 2 <--VariableArray
2 . 2 --->> Collected result of reading 2 from MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
   ... got measurement result. <<--- 2 . 2 <--VariableArray
10 . 2 --->> Collected result of reading 2 from MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
   ... got measurement result. <<--- 10 . 2 <--VariableArray
2 . 3 --->> Starting reading 3 on MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
   ... set up succeeded. <<--- 2 . 3 <--VariableArray
10 . 3 --->> Starting reading 3 on MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
   ... set up succeeded. <<--- 10 . 3 <--VariableArray
2 . 3 --->> Collected result of reading 3 from MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
   ... got measurement result. <<--- 2 . 3 <--VariableArray
2 --->> Finished all measurements from MeterTeros11 at SDI12-1_Pin7 , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 2 <--VariableArray
*****--- 4 sensors now complete ---***** <--VariableArray
10 . 3 --->> Collected result of reading 3 from MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
   ... got measurement result. <<--- 10 . 3 <--VariableArray
10 --->> Finished all measurements from MeterTeros11 at SDI12-6_Pin7 , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 10 <--VariableArray
2 --->> MeterTeros11 at SDI12-1_Pin7 powered down. <<--- 2 <--VariableArray
5 --->> BoschBME280 at I2C_0x76 powered down. <<--- 5 <--VariableArray
10 --->> MeterTeros11 at SDI12-6_Pin7 powered down. <<--- 10 <--VariableArray
*****--- 5 sensors now complete ---***** <--VariableArray
----->> Averaging results and notifying all variables. ... <--VariableArray
--- Averaging results from MeterTeros11 at SDI12-1_Pin7 --- <--VariableArray
--- Notifying variables from MeterTeros11 at SDI12-1_Pin7 --- <--VariableArray
--- Averaging results from BoschBME280 at I2C_0x76 --- <--VariableArray
--- Notifying variables from BoschBME280 at I2C_0x76 --- <--VariableArray
--- Averaging results from EnviroDIY Mayfly at EnviroDIY Mayfly --- <--VariableArray
--- Notifying variables from EnviroDIY Mayfly at EnviroDIY Mayfly --- <--VariableArray
--- Averaging results from MaximDS3231 at I2C_0x68 --- <--VariableArray
--- Notifying variables from MaximDS3231 at I2C_0x68 --- <--VariableArray
--- Averaging results from MeterTeros11 at SDI12-6_Pin7 --- <--VariableArray
--- Notifying variables from MeterTeros11 at SDI12-6_Pin7 --- <--VariableArray
... Complete. <<----- <--VariableArray

\/---- Line Saved to SD Card ----\/
2000-05-19 07:10:00,1.26399,22.10,0.000,21.28,43.707,101220.54,4.852,24.00,1.26615,21.60,0.000
GArrigotti-cws commented 3 years ago

Oddly enough, in if you put it into sensor testing mode now both sensors will work. We increased our wake time to five to ten seconds, but it will fail in normal logging interval. But will will allow the normal interval to work if you enable the variable array debug in the .ini file. @aufdenkampe will post my output- Still really odd.

aufdenkampe commented 3 years ago

@SRGDamia1, we've found a bug where we can only read 1 of 2 Teros11 sensors unless we have defined MS_VARIABLEARRAY_DEBUG. As soon as we do, both sensors read fine in the same sketch. When we don't define MS_VARIABLEARRAY_DEBUG, both sensors work on their own when the other is unplugged, but when both are plugged in, one or the other will not produce results.

Any ideas on what might be causing this bug?

I plan on working on this tomorrow (thanks @GArrigotti-cws for sending me two Teros11 sensors), so I could do specific testing.

SRGDamia1 commented 3 years ago

No, scanning all the debugging prints, I don't see anything in the variable array debug that would lead to that. It's particularly odd because if it was an error there, I'd guess we'd have seen it much earlier.

Do you have your SDI-12 addresses set one after the other or are they spaced out? I find sometimes the sensors get confused if I use "1" and "2" but putting space between them (ie "1" and "4") helps. I suspect that has something to do with the not-quite-exactly-perfect timing in the SDI-12 library due to bit-banging and interrupt hand-offs.

Could you please post turn on debugging for SDI-12 and the Terros and then post the full log from a couple of readings with and without the variable array debug? Obviously the output from the latter will be much longer than the former, but I should be able to line them up and see more of what's going on.

ie, first use and post output from :

-DMS_SDI12SENSORS_DEBUG
-DMS_SDI12SENSORS_DEBUG_DEEP
-DMS_METERTEROS11_DEBUG

then use and post output from:

-DMS_SDI12SENSORS_DEBUG
-DMS_SDI12SENSORS_DEBUG_DEEP
-DMS_METERTEROS11_DEBUG
-DMS_VARIABLEARRAY_DEBUG
GArrigotti-cws commented 3 years ago

@SRGDamia1 Here should be the information, I have the address one and six. I also placed sensor one at the top of the variable array and moved the other one to the bottom. @aufdenkampe and I did play with some of the timing issues also in varying configurations.

const char*   teros11SDI12address = "1"; 
const int8_t  terosPower = sensorPowerPin; 
const int8_t  terosData  = 7;              
const uint8_t teros11NumberReadings = 3;  

MeterTeros11 teros11(*teros11SDI12address, terosPower, terosData, teros11NumberReadings);

const char*   teros11_2SDI12address = "6"; 
MeterTeros11 teros11_2(*teros11_2SDI12address, terosPower, terosData, teros11NumberReadings);
> Executing task: C:\Users\ArrigottiG\.platformio\penv\Scripts\platformio.exe device monitor <

--- Available filters and text transformations: colorize, debug, default, direct, hexlify, log2file, nocontrol, printable, send_on_enter, time
--- More details at http://bit.ly/pio-monitor-filters
--- Miniterm on COM4  115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---

Now running Arrigotti_NoRadio.ino on Logger Arrigotti

Using ModularSensors Library version 0.28.4
Logger timezone is set to UTC -8
RTC timezone is set to UTC
Current RTC time is: 2000-06-06T03:48:12-08:00
All variable UUID's appear to be correctly formed.

2a5f077b-8de9-45ee-9704-4743726101fb -> SoilEa
168081de-2c1a-458f-8f86-1d88f7b8c394 -> SoilTemp
d7a3e8c8-d1f0-4503-a76f-93f5b6861850 -> SoilVWC
e5d304df-995b-4272-9a26-f3b525c315b7 -> BoschBME280Temp    
f8984e98-bbd4-4c29-a178-a64bcd546411 -> BoschBME280Humidity
6872a270-5dfb-4cee-9326-118adc97af9e -> BoschBME280Pressure
c3a10820-59ef-48a0-9fa5-88bb06914186 -> Battery
61e38606-b91c-4058-ac8c-3ff1ff280ede -> BoardTemp
29abed47-770f-44ef-893e-bff2645a1ff9 -> SoilEa
aa142833-d52e-4f91-b2b7-d3fa953acdae -> SoilTemp
49d31d59-1986-4a9c-ab6f-16d7639c8abd -> SoilVWC

This logger has a variable array with 11 variables, of which 11 come from 5 sensors and 0 are calculated.
Logger portion of setup finished.

Setting up sensors...
Enabling interrupts for SDI12 on pin 7 <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 did not reply! <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Getting sensor info <--SDI12Sensors
   SDI12 Address: 1 <--SDI12Sensors
   SDI12 Version: 1.30 <--SDI12Sensors
   Sensor Vendor: METER <--SDI12Sensors
   Sensor Model: TER11 <--SDI12Sensors
    Sensor Version: 112 <--SDI12Sensors
   Sensor Serial Number: T11-00006560 <--SDI12Sensors
  Suppressing DDI string on Meter sensor <--SDI12Sensors
Enabling interrupts for SDI12 on pin 7 <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Getting sensor info <--SDI12Sensors
   SDI12 Address: 6 <--SDI12Sensors
   SDI12 Version: 1.30 <--SDI12Sensors
   Sensor Vendor: METER <--SDI12Sensors
   Sensor Model: TER11 <--SDI12Sensors
    Sensor Version: 112 <--SDI12Sensors
   Sensor Serial Number: T11-00006574 <--SDI12Sensors
  Suppressing DDI string on Meter sensor <--SDI12Sensors
Setting up file on SD card
Data will be saved as Arrigotti_2000-06-06.csv
Putting processor to sleep

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 

The current clock timestamp is not valid!

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 
------------------------------------------
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors

 \/---- Line Saved to SD Card ----\/
2000-06-06 03:49:00,1.22331,21.50,0.000,23.24,45.878,101449.85,4.776,25.75,-9999.00000,-9999.00,0.000

------------------------------------------

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 

The current clock timestamp is not valid!

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 
------------------------------------------
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors

 \/---- Line Saved to SD Card ----\/
2000-06-06 03:50:00,1.22985,21.60,0.000,23.19,43.020,101448.22,4.745,26.00,-9999.00000,-9999.00,0.000

------------------------------------------

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 

The current clock timestamp is not valid!

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 
------------------------------------------
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors

 \/---- Line Saved to SD Card ----\/
2000-06-06 03:51:00,1.22309,21.60,0.000,23.19,42.824,101450.04,4.745,26.25,-9999.00000,-9999.00,0.000

------------------------------------------

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 

The current clock timestamp is not valid!

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 
------------------------------------------
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors

 \/---- Line Saved to SD Card ----\/
2000-06-06 03:52:00,1.22693,21.70,0.000,23.18,42.792,101448.77,4.761,26.75,-9999.00000,-9999.00,0.000

------------------------------------------

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 

The current clock timestamp is not valid!

----- WARNING ----- !!!!!!!!!!!!!!!!!!!!
!!!!!!!!!! ----- WARNING ----- !!!!!!!!!!
!!!!!!!!!!!!!!!!!!!! ----- WARNING ----- 
------------------------------------------
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
2 results expected This differs from the sensor's standard design of 3 measurements!!
    Concurrent measurement started. <--SDI12Sensors

 \/---- Line Saved to SD Card ----\/
2000-06-06 03:53:00,1.22668,21.80,0.000,23.25,42.572,101447.06,4.761,26.50,-9999.00000,-9999.00,0.000
SRGDamia1 commented 3 years ago

@GArrigotti-cws Could your turn on SDI-12 deep and Terrow (-DMS_SDI12SENSORS_DEBUG_DEEP -DMS_METERTEROS11_DEBUG) and post two different logs, running exactly the same program, one with -DMS_VARIABLEARRAY_DEBUG and one without?

GArrigotti-cws commented 3 years ago

@SRGDamia1 Here are all three separate logs.

build_flags =
    -DSDI12_EXTERNAL_PCINT
    -DNEOSWSERIAL_EXTERNAL_PCINT
    -DMQTT_MAX_PACKET_SIZE=240
    -DTINY_GSM_RX_BUFFER=64
    -DTINY_GSM_YIELD_MS=2
    -D MS_SDI12SENSORS_DEBUG    
    -DMS_SDI12SENSORS_DEBUG_DEEP 
    -DMS_METERTEROS11_DEBUG
    ;-DMS_VARIABLEARRAY_DEBUG

Deep and Terrow.txt

build_flags =
    -DSDI12_EXTERNAL_PCINT
    -DNEOSWSERIAL_EXTERNAL_PCINT
    -DMQTT_MAX_PACKET_SIZE=240
    -DTINY_GSM_RX_BUFFER=64
    -DTINY_GSM_YIELD_MS=2
    ;-D MS_SDI12SENSORS_DEBUG    
    ;-DMS_SDI12SENSORS_DEBUG_DEEP 
    ;-DMS_METERTEROS11_DEBUG
    -DMS_VARIABLEARRAY_DEBUG

Variable Array Debug.txt

build_flags =
    -DSDI12_EXTERNAL_PCINT
    -DNEOSWSERIAL_EXTERNAL_PCINT
    -DMQTT_MAX_PACKET_SIZE=240
    -DTINY_GSM_RX_BUFFER=64
    -DTINY_GSM_YIELD_MS=2
    ;-D MS_SDI12SENSORS_DEBUG    
    ;-DMS_SDI12SENSORS_DEBUG_DEEP 
    -DMS_METERTEROS11_DEBUG
    ;-DMS_VARIABLEARRAY_DEBUG

Just Meter Teros.txt

SRGDamia1 commented 3 years ago

@GArrigotti-cws I'm sorry, I really want two logs,

I want to see the raw SDI-12 commands and responses both with and without the variable array debug. What you posted with the variable array debug doesn't have the SDI-12 command/response, so I can't see what's different.

SRGDamia1 commented 3 years ago

Looking at the "Just Meter Teros" file, it looks like everything worked fine. Both sensors responded separately and you got different values in your csv for each.

GArrigotti-cws commented 3 years ago

@SRGDamia1 The other file you requested.

build_flags =
    -DSDI12_EXTERNAL_PCINT
    -DNEOSWSERIAL_EXTERNAL_PCINT
    -DMQTT_MAX_PACKET_SIZE=240
    -DTINY_GSM_RX_BUFFER=64
    -DTINY_GSM_YIELD_MS=2
    -D MS_SDI12SENSORS_DEBUG    
    -DMS_SDI12SENSORS_DEBUG_DEEP 
    -DMS_METERTEROS11_DEBUG
    -DMS_VARIABLEARRAY_DEBUG

Variable Array Debug.txt

Yeah, I noticed that. Not entirely sure why though, the same code is having very different outputs.

SRGDamia1 commented 3 years ago

The problem isn't with the sensors not responding, it's with the volumetric water content.

GArrigotti-cws commented 3 years ago

@SRGDamia1 But if I disable the debugging libraries then output is very, very different for the results saved to the SD Card. Same everything,

2000-06-06 05:39:00,1.25123,23.10,0.000,23.63,44.978,101405.84,4.882,24.75,-9999.00000,-9999.00,0.000 2000-06-06 05:40:00,1.25247,23.10,0.000,23.45,45.646,101403.51,4.852,24.75,-9999.00000,-9999.00,0.000 2000-06-06 05:41:00,1.24670,23.10,0.000,23.40,45.535,101404.56,4.852,24.75,-9999.00000,-9999.00,0.000

aufdenkampe commented 3 years ago

@GArrigotti-cws and @SRGDamia1, thanks for working on this!

Greg, Sara is asking for you to share with her the debug output when it fails. That's what she needs to see to figure out what is wrong.

Run it once with your platformio.ini file build flags like this, and save output to a text file called 4DebugFlags.txt.

build_flags =
    -D SDI12_EXTERNAL_PCINT
    -D NEOSWSERIAL_EXTERNAL_PCINT
    -D TINY_GSM_RX_BUFFER=64
    -D TINY_GSM_YIELD_MS=2
    -D MQTT_MAX_PACKET_SIZE=240
    -DMS_METERTEROS11_DEBUG
    -DMS_SDI12SENSORS_DEBUG 
    -DMS_SDI12SENSORS_DEBUG_DEEP
    -DMS_VARIABLEARRAY_DEBUG

Then run it again with the last line commented out, and save output to a text file called 3DebugFlags.txt.

build_flags =
    -D SDI12_EXTERNAL_PCINT
    -D NEOSWSERIAL_EXTERNAL_PCINT
    -D TINY_GSM_RX_BUFFER=64
    -D TINY_GSM_YIELD_MS=2
    -D MQTT_MAX_PACKET_SIZE=240
    -DMS_METERTEROS11_DEBUG
    -DMS_SDI12SENSORS_DEBUG 
    -DMS_SDI12SENSORS_DEBUG_DEEP
    ; -DMS_VARIABLEARRAY_DEBUG
SRGDamia1 commented 3 years ago

Ok, I have some suspicions about what might be causing the failures. I think it's probably something in the VWC calculation that's throwing everything else off. To narrow down the issue, I'm going to modify the Teros to output 4 variables - keeping the "calibratedCountsVWC" that is currently being ditched and I'm going to separate out the calculations a little. I'll try to get something ready for you to test tonight.

aufdenkampe commented 3 years ago

@SRGDamia1, thank you!

I'm on the phone with Greg right now, and he says that enabling -DMS_SDI12SENSORS_DEBUG_DEEP also causes both sensors to work. We had previously discovered that enabling -DMS_VARIABLEARRAY_DEBUG also causes both sensors to work.

So it makes it hard to see what is going on to cause the failures.

Sara, do you need more info about when these fail? I just had Greg edit his comments above to give you more info about which flags are on and which are off for each output.

GArrigotti-cws commented 3 years ago

@SRGDamia1 @aufdenkampe I can reproduce a failure by doing the following:

[build]
build_flags =
    -DSDI12_EXTERNAL_PCINT
    -DNEOSWSERIAL_EXTERNAL_PCINT
    -DMQTT_MAX_PACKET_SIZE=240
    -DTINY_GSM_RX_BUFFER=64
    -DTINY_GSM_YIELD_MS=2
    -D MS_SDI12SENSORS_DEBUG

Failure.txt

aufdenkampe commented 3 years ago

@SRGDamia1, Greg says the test sensors are in the air, so volumetric water content should be zero. That might not be the problem.

aufdenkampe commented 3 years ago

@SRGDamia1, I just noticed that you released Modular Sensors v0.28.5 two weeks ago.

Greg and I have been using v0.28.4. Any chance that might be a factor?

GArrigotti-cws commented 3 years ago

@aufdenkampe Looks like the latest modular sensors still did not rectify the problem, ironically it inverted which sensor had -9999.

2000-06-07 00:27:00,-9999.00000,-9999.00,0.000,23.13,44.730,101464.29,4.867,24.75,1.21920,23.10,0.000

My .ino file:

[external:libraries]
lib_deps =
    EnviroDIY_ModularSensors@=0.28.5
    https://github.com/PaulStoffregen/AltSoftSerial.git
    https://github.com/SRGDamia1/NeoSWSerial.git
    https://github.com/EnviroDIY/SoftwareSerial_ExternalInts.git
    https://github.com/vshymanskyy/StreamDebugger.git

[build]
build_flags =
    -DSDI12_EXTERNAL_PCINT
    -DNEOSWSERIAL_EXTERNAL_PCINT
    -DMQTT_MAX_PACKET_SIZE=240
    -DTINY_GSM_RX_BUFFER=64
    -DTINY_GSM_YIELD_MS=2
    -D MS_SDI12SENSORS_DEBUG

The .cpp file is as follows:


const char*   teros11SDI12address = "1"; 
const int8_t  terosPower = sensorPowerPin;  
const int8_t  terosData  = 7;            
const uint8_t teros11NumberReadings = 3;   

MeterTeros11 teros11(*teros11SDI12address, terosPower, terosData, teros11NumberReadings);

const char*   teros11_2SDI12address = "6"; 
MeterTeros11 teros11_2(*teros11_2SDI12address, terosPower, terosData, teros11NumberReadings);

These are the timings I tried with the new modular sensors:

#define TEROS11_WARM_UP_TIME_MS 250
#define TEROS11_STABILIZATION_TIME_MS 50
#define TEROS11_MEASUREMENT_TIME_MS 50
#define TEROS11_EXTRA_WAKE_TIME_MS 0
#define TEROS11_WARM_UP_TIME_MS 1550
#define TEROS11_STABILIZATION_TIME_MS 50
#define TEROS11_MEASUREMENT_TIME_MS 50
#define TEROS11_EXTRA_WAKE_TIME_MS 0

I verified against the new modular sensors, same results.

SRGDamia1 commented 3 years ago

Can you try the new sensor_calc branch: https://github.com/EnviroDIY/ModularSensors/tree/sensor_calc?

I added an internal parameter for sensors referring to the number of variables that are calculated internally and then split out the calculations for the Teros some. I also added some debugging.

Can you post results with no debugging and with all the SDI-12 debugging?

SRGDamia1 commented 3 years ago

you should be able to pull the new branch with:

lib_deps =
    https://github.com/EnviroDIY/ModularSensors.git#sensor_calc
GArrigotti-cws commented 3 years ago

@SRGDamia1 Would you like the deep debugging enabled? Or the normal SDI12?

GArrigotti-cws commented 3 years ago

@SRGDamia1 After some troubleshooting, the following output log was produced:

Sensor_Calc.txt

[external:libraries]
lib_deps =
    https://github.com/EnviroDIY/ModularSensors.git#sensor_calc
    https://github.com/PaulStoffregen/AltSoftSerial.git
    https://github.com/SRGDamia1/NeoSWSerial.git
    https://github.com/EnviroDIY/SoftwareSerial_ExternalInts.git
    https://github.com/vshymanskyy/StreamDebugger.git

[build]
build_flags =
    -DSDI12_EXTERNAL_PCINT
    -DNEOSWSERIAL_EXTERNAL_PCINT
    -DMQTT_MAX_PACKET_SIZE=240
    -DTINY_GSM_RX_BUFFER=64
    -DTINY_GSM_YIELD_MS=2
    -D MS_SDI12SENSORS_DEBUG    
    -DMS_SDI12SENSORS_DEBUG_DEEP 
2000-06-07 05:25:00,1.26650,22.20,0.000,22.60,41.964,101565.40,4.882,24.00,1.22685,22.40,0.000

Based on several outputs they both appear to be working with the above configuration. However, if I turn off the debugging:

    -D MS_SDI12SENSORS_DEBUG    
    -DMS_SDI12SENSORS_DEBUG_DEEP

Then I receive this output though I'm on the same branch:

2000-06-07 05:32:00,-9999.00000,-9999.00,-9999.000,22.69,41.926,101554.37,4.882,24.00,1.22309,22.50,0.000
SRGDamia1 commented 3 years ago

Okay. I'm flat-out stumped. I made some tiny changes, which you should be able to get with a pio lib update, but they shouldn't make a difference.

Get another Mayfly and program it with this freshly-written SDI-12 spy program: https://github.com/EnviroDIY/Arduino-SDI-12/blob/master/tools/SDI12_spy/SDI12_spy.ino

Connect the ground and sdi-12 data (7) pins, keeping your wires as short as possible. Post the output from the spy along with the output from running with and without the debugging flags. There probably will be some garbage in the output from the spy, especially if your grounds aren't perfect, but it should at least give us some idea of what's going on.

SRGDamia1 commented 3 years ago

There's definitely going to be garbage in the spy line, because it will parse the controlling line changes as garbage characters. I'll think about fixing that. But, anyway, even with garbage, it might help.

GArrigotti-cws commented 3 years ago

@SRGDamia1 I've got to buy some additional cables, but I'll do that tonight and attempt in the morning.

SRGDamia1 commented 3 years ago

https://github.com/EnviroDIY/Arduino-SDI-12/issues/77

GArrigotti-cws commented 3 years ago

@SRGDamia1 I'm hoping to get those cables to provide the information, I apologize for the delay. Had to shift priorities temporarily.

aufdenkampe commented 3 years ago

@SRGDamia1, we're back at this issue, and now I have two Teros11 sensors for testing.

For some reason, I'm seeing the error even when the relevant debug statements are turned on!

What I see is that within an action cycle, the first of the two SDI-12 sensors does not respond on the first try, but does on the second. We see this during setup with requestSensorAcknowledgement, but we try a second time and it works. However, we don't try to "Collect result of reading" a second time (i.e. the SDI12 command is sent only once per cycle). Therefore we fail.

This is the specific output sections I'm referring to:

9 . 1 --->> Starting reading 1 on MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< ␇6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 did not reply! <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors

Then later...

9 . 1 --->> Collected result of reading 1 from MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
MeterTeros11 at SDI12-6_Pin7 is reporting: <--MeterTeros11
    >>> 6D0! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
 <--SDI12Sensors
    <<< -9999.0000000000 <--MeterTeros11
    <<<

DETAILS:

The output below is compiled with:

build_flags =
    -D MS_SDI12SENSORS_DEBUG
    -D MS_SDI12SENSORS_DEBUG_DEEP
    -D MS_METERTEROS11_DEBUG
    -D MS_VARIABLEARRAY_DEBUG

lib_deps =
    https://github.com/EnviroDIY/ModularSensors.git#sensor_calc
Now running CWS-ThomasDairy1.ino on Logger CWS-ThomasDairy1

Using ModularSensors Library version 0.30.0
TinyGSM Library version 0.11.4

Logger timezone is set to UTC -8
RTC timezone is set to UTC
Current RTC time is: 2021-07-08T08:26:55-08:00
All variable UUID's appear to be correctly formed.

6b3a0e43-3392-4e83-b0aa-ed236b5d9ae9 -> Battery
db7d4a18-5b98-4188-bf7d-8762cb1da2d3 -> SoilEa
9da8d39f-710d-4678-b1f3-009f667a7073 -> SoilTemp
8c26d5d9-da81-4218-a198-a78c815cedce -> SoilVWC
fe553ef8-75cd-4677-9f01-82d9bbce3762 -> BoschBME280Temp
63401eba-984b-4b34-856d-9c0cbc9d9a48 -> BoschBME280Humidity
99b87310-03aa-40ca-beab-2696e3db22bb -> BoschBME280Pressure
858b111c-014b-40ad-bf24-4aa15c35f1ef -> SoilEa
929ca303-01b3-49a8-a96b-17121c8cb8c7 -> SoilTemp
f68e2659-f2bf-492e-a4d6-2313e825dc61 -> SoilVWC
c058446f-1f3f-4ece-b8a3-7cb88ef9953b -> signalPercent

There are 1 calculated variables in the group. <--VariableArray
There are 1 calculated variables in the group. <--VariableArray
This logger has a variable array with 11 variables, of which 10 come from 4 sensors and 1 are calculated.
Sampling feature UUID is: c24919f5-e95a-41ca-9bf8-ce241eeb0669
Logger portion of setup finished.

Setting up sensors...
Beginning setup for sensors and variables... <--VariableArray
Running sensor setup functions. <--VariableArray
    EnviroDIY Mayfly at EnviroDIY Mayfly was already set up! <--VariableArray
    Set up of MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
Enabling interrupts for SDI12 on pin 7 <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 1! <--SDI12Sensors
    <<< ␀e <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 did not reply! <--SDI12Sensors
    >>> 1! <--SDI12Sensors
    <<< 1 <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Getting sensor info <--SDI12Sensors
    >>> 1I! <--SDI12Sensors
    <<< 113METER   TER11 112T11-00011929 <--SDI12Sensors
   SDI12 Address: 1 <--SDI12Sensors
   SDI12 Version: 1.30 <--SDI12Sensors
   Sensor Vendor: METER <--SDI12Sensors
   Sensor Model: TER11 <--SDI12Sensors
    Sensor Version: 112 <--SDI12Sensors
   Sensor Serial Number: T11-00011929 <--SDI12Sensors
        ... setup succeeded. <--VariableArray
    Set up of BoschBME280 at I2C_0x76 ... <--VariableArray
        ... setup succeeded. <--VariableArray
    Set up of MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
Enabling interrupts for SDI12 on pin 7 <--SDI12Sensors
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Getting sensor info <--SDI12Sensors
    >>> 6I! <--SDI12Sensors
    <<< 613METER   TER11 114T11-00012692 <--SDI12Sensors
   SDI12 Address: 6 <--SDI12Sensors
   SDI12 Version: 1.30 <--SDI12Sensors
   Sensor Vendor: METER <--SDI12Sensors
   Sensor Model: TER11 <--SDI12Sensors
    Sensor Version: 114 <--SDI12Sensors
   Sensor Serial Number: T11-00012692 <--SDI12Sensors
        ... setup succeeded. <--VariableArray
... Success! <--VariableArray
Attempting to connect to the internet and synchronize RTC with NIST
This may take up to two minutes!
Could not wake modem for clock sync.
Putting modem to sleep
Setting up file on SD card
Data will be saved as CWS-ThomasDairy1_2021-07-08.csv
Putting processor to sleep

------------------------------------------
Creating a mask array with the uniqueness for each sensor.. <--VariableArray
Creating an array for the number of completed measurements.. <--VariableArray
Creating an array with the number of measurements to average.. <--VariableArray
Creating an array of the power pins.. <--VariableArray
Creating arrays of the power pin locations.. <--VariableArray
----->> Clearing all results arrays before taking new measurements. ... <--VariableArray
   ... Complete. <<----- <--VariableArray
----->> Powering up all sensors together. ... <--VariableArray
Powering up sensors... <--VariableArray
    Powering up EnviroDIY Mayfly at EnviroDIY Mayfly <--VariableArray
    Powering up MeterTeros11 at SDI12-1_Pin7 <--VariableArray
    Powering up BoschBME280 at I2C_0x76 <--VariableArray
    Powering up MeterTeros11 at SDI12-6_Pin7 <--VariableArray
   ... Complete. <<----- <--VariableArray
0 . 1 --->> Starting reading 1 on EnviroDIY Mayfly at EnviroDIY Mayfly ... <--VariableArray
   ... start reading succeeded. <<--- 0 . 1 <--VariableArray
0 . 1 --->> Collected result of reading 1 from EnviroDIY Mayfly at EnviroDIY Mayfly ... <--VariableArray
   ... got measurement result. <<--- 0 . 1 <--VariableArray
0 --->> Finished all measurements from EnviroDIY Mayfly at EnviroDIY Mayfly , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 0 <--VariableArray
0 --->> EnviroDIY Mayfly at EnviroDIY Mayfly powered down. <<--- 0 <--VariableArray
*****--- 1 sensors now complete ---***** <--VariableArray
6 --->> Waking BoschBME280 at I2C_0x76 ... <--VariableArray
   ... wake up success. <<--- 6 <--VariableArray
9 --->> Waking MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
   ... wake up success. <<--- 9 <--VariableArray
3 --->> Waking MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
   ... wake up success. <<--- 3 <--VariableArray
9 . 1 --->> Starting reading 1 on MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
    >>> 6C! <--SDI12Sensors
    <<< 600102 <--SDI12Sensors
    Concurrent measurement started. <--SDI12Sensors
   ... start reading succeeded. <<--- 9 . 1 <--VariableArray
3 . 1 --->> Starting reading 1 on MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 1! <--SDI12Sensors
    <<< 1 <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
    >>> 1C! <--SDI12Sensors
    <<< 100102 <--SDI12Sensors
    Concurrent measurement started. <--SDI12Sensors
   ... start reading succeeded. <<--- 3 . 1 <--VariableArray
9 . 1 --->> Collected result of reading 1 from MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
MeterTeros11 at SDI12-6_Pin7 is reporting: <--MeterTeros11
    >>> 6D0! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
 <--SDI12Sensors
    <<< -9999.0000000000 <--MeterTeros11
    <<< 
 <--SDI12Sensors
    <<< -9999.0000000000 <--MeterTeros11
Raw VWC Counts: -9999.00 <--MeterTeros11
Raw Temperature Value: -9999.00 <--MeterTeros11
WARNING:  raw results out of range (0-5000)!  Cannot calculate Ea or VWC <--MeterTeros11
WARNING:  Ea results out of range (0-350)!  Cannot calculate VWC <--MeterTeros11
WARNING:  temperature results out of range (-50-60)! <--MeterTeros11
   ... failed to get measurement result! <<--- 9 . 1 <--VariableArray
3 . 1 --->> Collected result of reading 1 from MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
MeterTeros11 at SDI12-1_Pin7 is reporting: <--MeterTeros11
    >>> 1D0! <--SDI12Sensors
    <<< 1 <--SDI12Sensors
    <<< + <--SDI12Sensors
    <<< 1812.9302000000 <--MeterTeros11
    <<< + <--SDI12Sensors
    <<< 26.4000000000 <--MeterTeros11
 <--SDI12Sensors
Raw VWC Counts: 1812.93 <--MeterTeros11
Raw Temperature Value: 26.40 <--MeterTeros11
Calculated Ea: 1.21 <--MeterTeros11
Calculated VWC: 1.21 <--MeterTeros11
Setting negative VWC to 0 <--MeterTeros11
   ... got measurement result. <<--- 3 . 1 <--VariableArray
9 . 2 --->> Starting reading 2 on MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< ␡6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 did not reply! <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
    >>> 6C! <--SDI12Sensors
    <<< 600102 <--SDI12Sensors
    Concurrent measurement started. <--SDI12Sensors
   ... start reading succeeded. <<--- 9 . 2 <--VariableArray
3 . 2 --->> Starting reading 2 on MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 1! <--SDI12Sensors
    <<< 1 <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
    >>> 1C! <--SDI12Sensors
    <<< 100102 <--SDI12Sensors
    Concurrent measurement started. <--SDI12Sensors
   ... start reading succeeded. <<--- 3 . 2 <--VariableArray
9 . 2 --->> Collected result of reading 2 from MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
MeterTeros11 at SDI12-6_Pin7 is reporting: <--MeterTeros11
    >>> 6D0! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
 <--SDI12Sensors
    <<< -9999.0000000000 <--MeterTeros11
    <<< 
 <--SDI12Sensors
    <<< -9999.0000000000 <--MeterTeros11
Raw VWC Counts: -9999.00 <--MeterTeros11
Raw Temperature Value: -9999.00 <--MeterTeros11
WARNING:  raw results out of range (0-5000)!  Cannot calculate Ea or VWC <--MeterTeros11
WARNING:  Ea results out of range (0-350)!  Cannot calculate VWC <--MeterTeros11
WARNING:  temperature results out of range (-50-60)! <--MeterTeros11
   ... failed to get measurement result! <<--- 9 . 2 <--VariableArray
3 . 2 --->> Collected result of reading 2 from MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
MeterTeros11 at SDI12-1_Pin7 is reporting: <--MeterTeros11
    >>> 1D0! <--SDI12Sensors
    <<< 1 <--SDI12Sensors
    <<< + <--SDI12Sensors
    <<< 1812.2201000000 <--MeterTeros11
    <<< + <--SDI12Sensors
    <<< 26.4000000000 <--MeterTeros11
 <--SDI12Sensors
Raw VWC Counts: 1812.22 <--MeterTeros11
Raw Temperature Value: 26.40 <--MeterTeros11
Calculated Ea: 1.20 <--MeterTeros11
Calculated VWC: 1.20 <--MeterTeros11
Setting negative VWC to 0 <--MeterTeros11
   ... got measurement result. <<--- 3 . 2 <--VariableArray
9 . 3 --->> Starting reading 3 on MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< ␡6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 did not reply! <--SDI12Sensors
    >>> 6! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
    MeterTeros11 at SDI12-6_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-6_Pin7 <--SDI12Sensors
    >>> 6C! <--SDI12Sensors
    <<< 600102 <--SDI12Sensors
    Concurrent measurement started. <--SDI12Sensors
   ... start reading succeeded. <<--- 9 . 3 <--VariableArray
3 . 3 --->> Starting reading 3 on MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
  Asking for sensor acknowlegement <--SDI12Sensors
    >>> 1! <--SDI12Sensors
    <<< 1 <--SDI12Sensors
    MeterTeros11 at SDI12-1_Pin7 replied as expected. <--SDI12Sensors
  Beginning concurrent measurement on MeterTeros11 at SDI12-1_Pin7 <--SDI12Sensors
    >>> 1C! <--SDI12Sensors
    <<< 100102 <--SDI12Sensors
    Concurrent measurement started. <--SDI12Sensors
   ... start reading succeeded. <<--- 3 . 3 <--VariableArray
9 . 3 --->> Collected result of reading 3 from MeterTeros11 at SDI12-6_Pin7 ... <--VariableArray
MeterTeros11 at SDI12-6_Pin7 is reporting: <--MeterTeros11
    >>> 6D0! <--SDI12Sensors
    <<< 6 <--SDI12Sensors
 <--SDI12Sensors
    <<< -9999.0000000000 <--MeterTeros11
    <<< 
 <--SDI12Sensors
    <<< -9999.0000000000 <--MeterTeros11
Raw VWC Counts: -9999.00 <--MeterTeros11
Raw Temperature Value: -9999.00 <--MeterTeros11
WARNING:  raw results out of range (0-5000)!  Cannot calculate Ea or VWC <--MeterTeros11
WARNING:  Ea results out of range (0-350)!  Cannot calculate VWC <--MeterTeros11
WARNING:  temperature results out of range (-50-60)! <--MeterTeros11
   ... failed to get measurement result! <<--- 9 . 3 <--VariableArray
9 --->> Finished all measurements from MeterTeros11 at SDI12-6_Pin7 , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 9 <--VariableArray
*****--- 2 sensors now complete ---***** <--VariableArray
3 . 3 --->> Collected result of reading 3 from MeterTeros11 at SDI12-1_Pin7 ... <--VariableArray
MeterTeros11 at SDI12-1_Pin7 is reporting: <--MeterTeros11
    >>> 1D0! <--SDI12Sensors
    <<< 1 <--SDI12Sensors
    <<< + <--SDI12Sensors
    <<< 1813.7501000000 <--MeterTeros11
    <<< + <--SDI12Sensors
    <<< 26.4000000000 <--MeterTeros11
 <--SDI12Sensors
Raw VWC Counts: 1813.75 <--MeterTeros11
Raw Temperature Value: 26.40 <--MeterTeros11
Calculated Ea: 1.22 <--MeterTeros11
Calculated VWC: 1.22 <--MeterTeros11
Setting negative VWC to 0 <--MeterTeros11
   ... got measurement result. <<--- 3 . 3 <--VariableArray
3 --->> Finished all measurements from MeterTeros11 at SDI12-1_Pin7 , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 3 <--VariableArray
*****--- 3 sensors now complete ---***** <--VariableArray
6 . 1 --->> Starting reading 1 on BoschBME280 at I2C_0x76 ... <--VariableArray
   ... start reading succeeded. <<--- 6 . 1 <--VariableArray
6 . 1 --->> Collected result of reading 1 from BoschBME280 at I2C_0x76 ... <--VariableArray
   ... got measurement result. <<--- 6 . 1 <--VariableArray
6 . 2 --->> Starting reading 2 on BoschBME280 at I2C_0x76 ... <--VariableArray
   ... start reading succeeded. <<--- 6 . 2 <--VariableArray
6 . 2 --->> Collected result of reading 2 from BoschBME280 at I2C_0x76 ... <--VariableArray
   ... got measurement result. <<--- 6 . 2 <--VariableArray
6 . 3 --->> Starting reading 3 on BoschBME280 at I2C_0x76 ... <--VariableArray
   ... start reading succeeded. <<--- 6 . 3 <--VariableArray
6 . 3 --->> Collected result of reading 3 from BoschBME280 at I2C_0x76 ... <--VariableArray
   ... got measurement result. <<--- 6 . 3 <--VariableArray
6 --->> Finished all measurements from BoschBME280 at I2C_0x76 , putting it to sleep. ... <--VariableArray
   ... succeeded in putting sensor to sleep. <<--- 6 <--VariableArray
3 --->> MeterTeros11 at SDI12-1_Pin7 powered down. <<--- 3 <--VariableArray
6 --->> BoschBME280 at I2C_0x76 powered down. <<--- 6 <--VariableArray
9 --->> MeterTeros11 at SDI12-6_Pin7 powered down. <<--- 9 <--VariableArray
*****--- 4 sensors now complete ---***** <--VariableArray
----->> Averaging results and notifying all variables. ... <--VariableArray
--- Averaging results from EnviroDIY Mayfly at EnviroDIY Mayfly --- <--VariableArray
--- Notifying variables from EnviroDIY Mayfly at EnviroDIY Mayfly --- <--VariableArray
--- Averaging results from MeterTeros11 at SDI12-1_Pin7 --- <--VariableArray
--- Notifying variables from MeterTeros11 at SDI12-1_Pin7 --- <--VariableArray
--- Averaging results from BoschBME280 at I2C_0x76 --- <--VariableArray
--- Notifying variables from BoschBME280 at I2C_0x76 --- <--VariableArray
--- Averaging results from MeterTeros11 at SDI12-6_Pin7 --- <--VariableArray
--- Notifying variables from MeterTeros11 at SDI12-6_Pin7 --- <--VariableArray
... Complete. <<----- <--VariableArray

 \/---- Line Saved to SD Card ----\/
2021-07-08 08:30:00,4.867,1.20816,26.40,0.000,25.00,44.382,97955.52,-9999.00000,-9999.00,-9999.000,-9999
aufdenkampe commented 3 years ago

I found the fix. 

build_flags =
    -D MS_SDI12_NON_CONCURRENT

Evidently the Teros 11 sensors do not like concurrent SDI-12 measurements. Both of my sensors are supposedly SDI12 Version: 1.30 but they do have different Meter firmware versions (112 & 114).

This works with or without the debugging statements turned on. ... when using the sensor_calc branch at https://github.com/EnviroDIY/ModularSensors/commit/a56429d40f531cfb25016652594759d1d00ea230

aufdenkampe commented 3 years ago

... but defining MS_SDI12_NON_CONCURRENT does't work for the new ModularSensors v0.30.0 or for v0.28.5! In fact, neither sensor reports, so the situation is worse.

I'm beginning to think that the solution will require modification of the getResults() function in src/sensors/SDI12Sensors.cpp to add a multiple tries loop similar to what is in the requestSensorAcknowledgement() function. Specifically, something starting like this:

while (!didAcknowledge && ntries < 5) {
        _SDI12Internal.sendCommand(myCommand, _extraWakeTime);

This is slightly complicated, however, because there is already a while loop there, which I don't want to mess up, nor the stream buffer for those subsequent functions.

aufdenkampe commented 2 years ago

@SRGDamia1, thanks for merging master into the sensor_calc branch. I'm interested in revisiting and finally fixing this issue over the next month or two.

From my comment above, I may need your help in implementing the fix. I'll let you know when I'm ready for testing again.

SRGDamia1 commented 2 years ago

@aufdenkampe - do let me know, but I'm really, really behind on pretty much everything right now. Before I left in July, I was starting to pick through the SDI-12 library to see if I could optionally use a different (10 or 16 bit) timer where possible instead of the 8bit timer to be able to have a little more buffer in reading the SDI-12 bits in before the timer rolls over. I was thinking that might improve the back and forth just a touch. But I haven't picked it up since then and I don't know when I'll get a chance to think about it again.

I've also dropped my hours just a little to better manage 4 kids at home.