DS18b20 max number of sensors and frame size

ArcticSnow commented 5 years ago

There are two problems with the current handling for DS18b20: 1- there seem to be a limit on how many sensor we can sample with the current version of the code. Fro instance, we built two lines of 21 sensor each. We could read both line independently just fine but when both are plugged into the same pin, finding their address works but reading the data fails. The same two line on the same pin can be read using the exmaple script (looping throuhg each sensor one by one). When connecting the two line on two indenpendt digital pin, the code can fine the 42 sensor address, but only reads data from 40 sensors.

2- When we have a large number of sensors, we can overfill the frame, and it fails to create it. The code could be revised to split the array in 2 to make sure we never overfill frames.

jdavid commented 5 years ago

There was a limit of 40 sensors actually.. raised that to 99

jdavid commented 5 years ago

Two more changes:

If the sensor doesn't fit in the current frame, send it in another frame.
Send the name only in the first (boot) frame, only enabled with Xbee since (it would introduce a bug in 4G)

Now in the server it's possible to have 2 frames from the same device with the same time, if they have a different sequence number.

With these 2 changes the 42 string should fit.

jdavid commented 5 years ago

The size of a binary frame with a DS18B20 string of 42 sensors:

Header: 15 bytes (the node id will always be empty now)
TST field: 5 bytes
DS18B20 field : 3 + 42

Total is 65 bytes. Since the frame size is 73 bytes, this means there're still 8 bytes left.

The 42 number actually is variable. Every sensor is stored in 1 byte, if the temperature difference from the previous sensor fits in 1 byte, otherwise it will use 2 bytes. This means we can have up to 8 big changes in temperature in the string, if there're more the field won't fit in the frame.

A big change in temperature is 8 degrees (128 / 16), so we can fit a 42 string in the frame if there're no more than 8 changes of 8 degrees in the string. My impression is that this is very unlikely to happen, correct me if I'm wrong, so I'm closing this issue.

In my opinion we should move away from the DigiMesh to a solution that works at 868Mhz and has a frame size larger than 73 bytes (some other network modules have 100 bytes), but that would be a different issue.

ArcticSnow commented 5 years ago

I think this issue is not fully debuged. After pushing the script as of today (Jan 21, 2019), to the waspmote in the marsh connected to one ctd10 and two strings of 20 ds18b20 each, I got the following:

==========================================
1548080313.713 INFO Id        : 556667057C10542D Version=H Name=fw-mwtab
1548080313.737 INFO Battery   : Lithium-ion (100 %)
1548080313.757 INFO Hardware  : board=lemming SD=1 GPS=1
1548080313.779 INFO XBee      : mac=0013A200416A0720 hw=184D sw=80670000 network="Finse"
1548080313.805 INFO Frames    : size=73 encryption=disabled
1548080313.827 INFO Log       : level=DEBUG output=USB, SD
1548080313.852 INFO Actions   : net=1h bat=5m ctd=5m ds1820=1m 
1548080313.876 INFO Boot done, go to sleep
1548080313.939 DEBUG Next alarm at 00:14:19:00 in 1 minutes
1548080340.873 INFO *** Loop 1 battery=Lithium-ion (100 %)
1548080342.310 DEBUG OneWire(2) 28605B860A000082  : -14.12
1548080342.349 DEBUG OneWire(2) 2850E2850A00005B  : -2.37
1548080342.389 DEBUG OneWire(2) 2888CF850A00002A  : -11.18
1548080342.429 DEBUG OneWire(2) 2818CF850A00009B  : -13.25
1548080342.469 DEBUG OneWire(2) 289889850A000004  : -7.50
1548080342.513 DEBUG OneWire(2) 28648E850A00005A  : -5.12
1548080342.553 DEBUG OneWire(2) 2854D5850A000052  : -4.06
1548080342.593 DEBUG OneWire(2) 284C8E850A00004D  : -3.25
1548080342.633 DEBUG OneWire(2) 280272860A000024  : -1.81
1548080342.673 DEBUG OneWire(2) 28D293850A00007F  : -0.87
1548080342.713 DEBUG OneWire(2) 2872D2850A000007  : -0.37
1548080342.753 DEBUG OneWire(2) 28C691850A00007B  : -13.00
1548080342.794 DEBUG OneWire(2) 280E97850A0000D9  : -13.18
1548080342.835 DEBUG OneWire(2) 284130860A000062  : -13.37
1548080342.880 DEBUG OneWire(2) 28C991850A00005F  : -13.43
1548080342.920 DEBUG OneWire(2) 28F9D3850A0000D8  : -13.37
1548080342.959 DEBUG OneWire(2) 28ADB8850A00004B  : -13.56
1548080342.999 DEBUG OneWire(2) 286B3E860A0000B9  : -13.81
1548080343.039 DEBUG OneWire(2) 28C788850A0000C3  : -13.43
1548080343.079 DEBUG OneWire(2) 28B7F3850A000046  : -13.37
1548080343.119 DEBUG OneWire(2) 286FD5850A000047  : -13.87
1548080343.961 DEBUG OneWire(6) 2860D5850A000063  : -13.75
1548080344.004 WARN OneWire(6) 2888FC850A0000E0 bad data, CRC failed: FFFFFFFFFFFFFFFF C9
1548080344.049 DEBUG OneWire(6) 28A489850A000094  : -6.31
1548080344.089 DEBUG OneWire(6) 2854F6850A0000E4  : -4.56
1548080344.129 DEBUG OneWire(6) 288C4D860A000036  : -3.37
1548080344.169 DEBUG OneWire(6) 289C6E860A0000DB  : -2.93
1548080344.209 DEBUG OneWire(6) 280288850A0000B7  : -1.62
1548080344.249 DEBUG OneWire(6) 28D2F1850A0000ED  : -1.37
1548080344.290 DEBUG OneWire(6) 28B2C3850A000029  : -0.75
1548080344.330 DEBUG OneWire(6) 2886DB850A00005A  : -12.68
1548080344.374 DEBUG OneWire(6) 280ED5850A0000B3  : -12.93
1548080344.415 DEBUG OneWire(6) 2801F4850A0000A2  : -13.00
1548080344.455 DEBUG OneWire(6) 2849F9850A00009A  : -13.06
1548080344.495 DEBUG OneWire(6) 2879D5850A0000AE  : -13.18
1548080344.536 DEBUG OneWire(6) 280DCA850A0000F9  : -13.06
1548080344.576 DEBUG OneWire(6) 2843CC850A000003  : -13.31
1548080344.618 DEBUG OneWire(6) 284BCD850A00006F  : -13.37
1548080344.658 DEBUG OneWire(6) 2847D5850A000050  : -13.37
1548080344.703 DEBUG OneWire(6) 2857D5850A00000B  : -13.31
1548080344.743 DEBUG OneWire(6) 280FCF850A000045  : -9.68
1548080344.783 DEBUG OneWire(6) 286FE8850A00002F  : -13.06
1548080344.805 WARN No space left in frame
1548080344.826 WARN No space left in frame
1548080344.848 INFO Loop done in 4848 ms (CPU time 4848 ms).
1548080344.913 DEBUG Next alarm at 00:14:20:00 in 1 minutes

Could the failing sensor mess with what is built in the frame?

jdavid commented 5 years ago

Should not. I've counted at most 6 changes including that CRC error, it should fit. So maybe I've done some mistake in the calculation. But still 6 is pretty close to 8. I'll split the field in 2 to be safer.

jdavid commented 5 years ago

Now the field will be split in chunks of 30 values at most. I still have to do the parsing side in the Pi/Server.

jdavid commented 5 years ago

Have added a new command, merge_frames.py, to merge the split frames that were already in the database, there were 1.852

Also changed the code so these frames will be merged as they are created.

spectraphilic / wasp_sketches

DS18b20 max number of sensors and frame size #35