Switch examples to use Digi LTE-M Transparent mode

[aufdenkampe]

It's probably time to switch our examples to use Digi XBee3 LTE-M radio module in transparent mode. Presently, only the DRWI_LTE.ino example uses transparent mode, and that approach seems to be working well for a number of us.

Background:

Our new LTEbee-Adapter allows us to use Transparent mode for the Digi XBee3 LTE-M module, without brown-outs. Previously, we could only get Bypass mode to work well, likely because of less stringent power demands. In fact, with the current version of the library (v0.23.4) it now appears that Transparent mode is more stable than Bypass mode, and recommended, even thought it still has some draw backs.

@SRGDamia1 summarized her latest findings here: https://github.com/EnviroDIY/ModularSensors/commit/1f8615dffefea1ffe592ac4f90092fd78880b724#commitcomment-34592670

At the beginning of the summer I finally got some boards and SIM cards to test the LTE a bit more "long term" on my desk. Running in bypass mode seemed to send the data more consistently and to use up less battery. I was all ready to confirm that bypass was better, when the board completely locked up - all the lights on and doing nothing. So I reprogrammed it to turn on more of the debugging and let it go again.. and it locked up after a few days. I could never get it to lock up "on queue" and still can't figure out why it's locking up. Every time, hitting the reset button magically fixes all ills. The transparent board never locked up, just ran out of battery faster and had less success in sending data. I decided that was a better trade off. In the process I also figured out and implemented a watch dog timer to force a reset in event of a freeze up while the power is on.

Summary: Transparent + hasn't been observed to cause mcu to fully lock up + uses very slightly less data with every data transmission - doesn't make a connection as quickly - uses more battery power because of slower connection - completely misses more data points (you can see many of them on hologram as 0-data connections)

Bypass + connects faster = lower power + almost every point makes it to the web - uses slightly more data with each individual transmission (this is in overhead we cannot control) - sometimes causes everything to freeze up while fully powered - relies on the watch-dog timer to prevent full shutdown.

Considering my positive experiences deploying with Transparent mode in recent weeks, and comments from @neilh10 in https://github.com/EnviroDIY/ModularSensors/commit/1f8615dffefea1ffe592ac4f90092fd78880b724#commitcomment-34697982, let's make the switch in our examples and docs.

[neilh10]

Thanks for the references to the LTEbee-adapter, I hadn't seen the git, I've got an adapter on the way. I've started using the DigiXBeeCellularTransparent/Mayfly with Digi LTE CatM1 XB3-C-A2 - with a hologram SIM - though working on trying to get it provision with the SIM.

One item I did for a piece of production equipment testing - was to read and print the SIM #. This verifies the basics all work - from Digi module seated to small SIMs readable.

[SRGDamia1]

@aufdenkampe can you do this? It might be a bit before I could get to it.

[aufdenkampe]

@SRGDamia1, yes, I'll give it a go. I need to code up some stations today for a new group, and it would be nice to issue a new release.

Thank your for accepting those suggestions and PRs from @neilh10.

[neilh10]

FYI - another area that changed with "0.23.4" from 0.23.2 is time adjustments - and seems to have broken - though 1) time offsets are always challenging 2) I've been running into testing challenges getting to the Time Server as with https://github.com/EnviroDIY/ModularSensors/issues/281

So just referencing ModularSensors\examples\menu_a_la_carte in setup() this changed

``` Logger::setLoggerTimeZone(timeZone); // It is STRONGLY RECOMMENDED that you set the RTC to be in UTC (UTC+0) Logger::setRTCTimeZone(0); ```

which I totally agree that the internal source of RTC should always be UTC+0. IMHO it should be removed as an option in any example for the internal RTC synced with NIST to be anything else other than the UTC+0 supplied by NIST.
When there is human display elements needed then it is as what is currently done using UTC+TZ and as per https://en.wikipedia.org/wiki/ISO_8601

I'm finding some challenge in working through the code in LoggerBase.cpp which has some good changes to names, but still being used in the old way. So the new/refactored LoggerBase.h protected: // Static variables - identical for EVERY logger static int8_t _loggerTimeZone; static int8_t _loggerRTCOffset;

that _loggerRTCOffset; contains the values that should be used in Offset calculations. "_loggerTimeZone" is a label that references the time zone and from my understanding should never be used in offset calculations, which it is in setRTClock()

For current purposes _loggerRTCOffset; is the same as _loggerTimeZone - but if there where adjustments built infor geographical based SummerTime switches, then _loggerRTCOffset changes but not _loggerTimeZone.

So I'm changing LoggerBase ::setRTClock so that only uses _loggerRTCOffset in offset calculations; and that I can test.

I could submit it when tested in my envr.?

[aufdenkampe]

@neilh10, that would be great. I have indeed found that the RTC sync doesn't seem work for me.

[SRGDamia1]

_loggerTimeZone and _loggerRTCOffset are not intended to change. _loggerTimeZone is the timezone to print out in the file and _loggerRTCOffset is the difference between the printed time and the actual value on the DS3231. I'm not sure why when I set it up initially I did it as the printed time and then the offset instead of the printed time and the real running time. It was all working as is until someone from New Zealand found that the couldn't use offsets over nine, which was when I made some tweaks - mostly renaming things. Because the subtraction was working, I left it as _loggerTimeZone and _loggerRTCOffset.

There never was (nor do I ever plan to add) support for daylight savings time of any kind. I HATE HATE daylight savings time. I'm fine with calling noon 1pm, but dealing with the change mid year has caused me to lose truly enormous amounts of time in trying to figure out what happened when one device decided to apply DST and another didn't. There's no excuse for it. I hate daylight savings time.

The time sync is definitely not perfect, but I think the errors are all on the end of making the connection with the daylight server and being able to read its time before NIST closes the connection. All of the modems struggle with it, but the XBee is the worst. Often the modem spits out the "closed" message without ever reporting data being received. I think I've had the most luck with the ESP8266 (wifi) and the SIM800 was in the middle.

[aufdenkampe]

@SRGDamia1, I think we all agree that we don't wan't to deal with daylight savings time, and always use standard time.

If I'm understanding the rest of your response, it sounds like things are as they should be? I admit that I haven't dug into those code changes, but just applied them as you changed them in the examples, and they have all be working well for me.

[SRGDamia1]

@aufdenkampe - yes, I think all is working as-is.

There are three "time-zone-related" bits of information - the timezone of the DS3231, the timezone that is being printed to files and any other outputs, and the offset between these two. Obviously, the library only needs to store two of them and can always calculate the third. I think what @neilh10 is getting at is that the setRTClock and a few other functions are un-necessarily complicated by the fact that for reasons long forgotten I chose to store one timezone and the offset and calculate the second timezone every time instead of storing both timezones.

Neil also mentioned:

For current purposes _loggerRTCOffset; is the same as _loggerTimeZone - but if there where adjustments built infor geographical based SummerTime switches, then _loggerRTCOffset changes but not _loggerTimeZone.

So I wanted to make it clear that I don't ever plan to deal with any summer-time switches or support code to deal with them.

[neilh10]

I totally agree. Internal RTC always UTC. I've never done summer time - California had an initiative a few years ago to be able to not change. I've effectively only implemented _loggerRTCOffset in code - so was interested in ISO_8601 and the way that it was supported in LoggerBase. So _loggerRTCOffset should only used in calculations. _loggerRTCOffset used in printouts

[aufdenkampe]

@SRGDamia1, for DigiXBeeCellularTransparent, should useCTSforStatus be true or false?

In your DRWI_LTE.ino you have on line 87:

const bool useCTSforStatus = false;   // Flag to use the modem CTS pin for status

But in the menu_a_la_carte.ino, in the remarked out code on line 206 you have it set to true.

[SRGDamia1]

That flag depends on your hardware connection. You just have to know what you have connected.

If you are using a Mayfly without an LTE adapter board, then it's true, because pin 19 of the mayfly (that you set as the status pin) is connected to Bee pin 12, which is CTS_not. If you have a Mayfly with an adapter board, it's false.

[SRGDamia1]

You need the flag because CTS_not will be HIGH when the Bee is asleep (ie, it's not clear to send) while Status/Sleep_not will be LOW when the Bee is asleep.

The Mayfly was designed around the Sodaq GPRSBee, which broke out the SIM800's status pin to the XBee CTS pin instead of to its status pin.

[aufdenkampe]

@SRGDamia1, thanks for that info. I just changed the menu_a_la_carte.ino. Give it a quick glance to see if it looks right, and I'll replicate, merge to master and issue a release.

[SRGDamia1]

@aufdenkampe - FYI, while I haven't been able to figure out why the bypass mode sometimes completely freezes up the board, the watch-dog timer does successfully restart the board when it is frozen. It's probably a bit more risky depending on the watch-dog timer like that, but on the other hand, that's exactly what the watch-dog is designed for.

So, you're trading dependence on the watch-dog timer and higher data use for more successful transmissions and less battery draw - not freezes completely vs doesn't freeze up. For the ones we've got going out this week, @s-hicks2 thinks it's probably worth it.

I haven't run a long enough test to see the watch-dog come into effect for other reasons, but I'm hoping that it will reduce the number of times that stations need to be visited and power cycling them is a magic fix.