CRC Validation for SDI12 v1.4

[KareemWaheed]

Hi,

Currently, I'm using Metergroup SDI-12 soil sensors (Teros 11 - Teros 12)

Using examples to get measurements from the sensor, I sometimes have some “bad” values returning -68 for soil moisture and 0 for temperature. Is it usual to have some “errors” during measurement with the Sensor like Teros 11-12? Is it a bad initialization? I could repeat measurement until a value is different from -68 in my code, but this is a trick, and it will just mask the real problem.

[SRGDamia1]

I'm only vaguely familiar with that sensor.

Are you seeing this only when using this SDI-12 library or are you seeing it with non-Arduino data loggers? If the latter, its a sensor error.

To get a value of -68 the sensor has to have responded to the Arduino. If there was no response, the value should be -9999. It's possible that the value is being misread, but I would have expected that if the value was being misread between the Arduino and the sensor the response would be variable, not always the same value. Which, again, makes me suspicious of an instrument problem.

You can try adding a short (<100ms) "wake delay" to commands to give the logger a bit more time to respond. (use sendCommand(command, delay);) See if that reduces errors.

[KareemWaheed]

I'm only vaguely familiar with that sensor.

Are you seeing this only when using this SDI-12 library or are you seeing it with non-Arduino data loggers? If the latter, its a sensor error.

To get a value of -68 the sensor has to have responded to the Arduino. If there was no response, the value should be -9999. It's possible that the value is being misread, but I would have expected that if the value was being misread between the Arduino and the sensor the response would be variable, not always the same value. Which, again, makes me suspicious of an instrument problem.

You can try adding a short (<100ms) "wake delay" to commands to give the logger a bit more time to respond. (use sendCommand(command, delay);) See if that reduces errors.

Unfortunately i don't have any datalogger to test the issue The issue happened with more than one sensor so i'm suspecting my code or the nodemcu part

Actually the -68 is after using the equation which is (3.879 (10^-4) reading - 0.6956)*100

Which make the value the nodemcu received around 0 more or less which is weird

[SRGDamia1]

Again, it sounds like an instrument error, not anything with this library. If the sensor wasn't responding, you'd get -9999 or something calculated from that. The library is built to ensure that the value returned from a read timeout is not 0. If you are consistently parsing the same value from the sensor's response, it seems like the sensor is really just intermittently spitting out 0's instead of real values. You can try using a different Arduino mcu (ie, an Uno instead of an ESP) as an alternate logger to cross-check if you have one of those, but I doubt you'd see anything different.

A few years ago I did have someone report something similar on a different library with the prior version of this sensor, but I think the "fix" I ended up implementing in that case was to re-read when I got bad values: https://github.com/EnviroDIY/ModularSensors/issues/135

[aufdenkampe]

@KareemWaheed, I suspect that the Teros 11 sensor is getting poor contact with the soil media, and therefore giving a "0" for the raw permittivity value, that would then translate to a calculated -68% volumetric soil moisture.

Note that we did have some timing issues when incorporating the Teros 11 in the Modular Sensors library: https://github.com/EnviroDIY/ModularSensors/issues/276. This issues aren't totally resolved.

[KareemWaheed]

@SRGDamia1 @aufdenkampe Thanks for your help We managed to overcome this issue by validating CRC

with a Function divided entire response into 2 separate strings: last 3 characters are put into "RecCRC", and everything before that is put into "RecString". The CRC is calculated again, taking "RecString" as input, using the CRC function mentioned in the SDI12 specs document (V. 1.4) in page 21, and output result is saved in the string named "Result". "Result" and "RecCRC" are compared. If there is an exact match between them, then the result is accepted (and the function returns false).

bool CRCCalc(String EntireResp){

  uint8_t NChar = EntireResp.length()-2; //number of characters without <CR> and <LF> (readable string composed of sensor address, values separated by + and -) and the 3 characters
  uint16_t CRC = 0;
  String Result = "";
  String RecCRC = "";
  String RecString = "";
  for(int i = (NChar-3); i<(NChar); i++) RecCRC += EntireResp[i]; //extract the 3 characters from the string
  //Serial.println(RecCRC);
  for(int i = 0; i<(NChar-3); i++) RecString += EntireResp[i];
  //Serial.println(RecString);

  for(int i = 0; i<RecString.length(); i++){
        CRC ^= RecString[i];
        for(int j=0;j<8;j++){
            if(CRC & 0x0001){
                CRC >>= 1;
                CRC ^= 0xA001;
            }
            else{
                CRC >>= 1;
            }
        }
    }
    Result += (char)(0x40 | (CRC>>12));
    Result += (char)(0x40 |((CRC>>6)&0x3F));
    Result += (char)(0x40 | (CRC &0x3F));

    if(RecCRC == Result){
     // Serial.println("Match");
      return false;
    }
    else{
      // Serial.println("Didn't Match");
      return true;
    }
}

late better than never 😆

[aufdenkampe]

@KareemWaheed, thank you for sharing that back with us! That is very helpful to learn that validating CRC was the main issue. Thanks also for the function that you used to calculate the CRC!

[aufdenkampe]

@SRGDamia1, did you ever get a chance to incorporate the SDI12 CRC validation fix described by @KareemWaheed in https://github.com/EnviroDIY/Arduino-SDI-12/issues/75#issuecomment-1153129364 above?

[SRGDamia1]

@aufdenkampe No, I haven't done anything with the CRC. There's another outstanding PR (#82) related to CRC implementation.

[SRGDamia1]

@aufdenkampe checking the received CRC would need to go into the logger code (ie, ModularSensors) not here. The other PR is related to implementing a CRC when using the Arduino as an SDI-12 slave.