Added FDRS Timekeeping

[aviateur17]

You can reject if you want to add this to a different branch in order to wait to include in the main release branch. This adds code in Gateway for NTP Time server queries via WiFi. Compiled in Arduino 1.8.19 with and without USE_WIFI defined.

[aviateur17]

... predictably "esp_err_t" isn't defined for it (for starters).

I think I have the fix for that. I found that as well and instead of using another struct for esp_err_t I will change that to a typdef as below and that will work for other micros.

#ifndef ESP32
typedef int esp_err_t;
#define ESP_FAIL 0
#define ESP_OK 1
#endif

I think that it would be okay to use #ifdefs to switch between chip-specific RTC routines. This might come up again when adding support for varying external RTC modules. If there is a library needed, it would be okay to require it for timekeeping, but my hangup is having it required to install for users who may not need timekeeping.

Looks like AVR doesn't have "sys/time.h" either.

For now, should we stipulate that timekeeping is only for Espressif chips? Have you confirmed it on anything else?

I've only confirmed on ESP32 and ESP8266. I have a Sparkfun ProRF LoRa which has SAMD21 on it but I haven't yet put FDRS on that one. I think we should say that it's only been tested on Espressif chips and would welcome other feedback.

I have an MQTT gateway with an ESP-NOW repeater. The MQTT gateway gets the accurate time and gives it to the repeater. However, the repeater will then end up giving its skewed time back to the MQTT gateway.

Yeah, right now the time is sent via ESP-NOW back to both neighbors and all registered nodes. We can change that if we figure out a better method.

What if instead of just sharing the time periodically, it was restricted to only happening when the gateway is given a verified-accurate time? So devices would only share the time when they receive a "fresh" time, either by NTP or a gateway sending one. This applies for sharing amongst gateways... nodes would be different and I haven't gotten to them.

Yes, we can only have the time sent when it is refreshed via NTP. That won't currently prevent it being sent from the ESP-NOW repeater back to the MQTT gateway in your example above though. I could implement that if the

#define TIME_SEND_INTERVAL 5    // Time, in minutes, between sending out time to remote devices

Is set to 0 then only send the time when received by NTP and we can have the default value at 0. However with introduction of RTC without NTP we will probably need the periodic send. The other thing is that if, for some reason, the gateways miss the packet being transmitted at the time they won't get the new time for potentially quite awhile.

[aviateur17]

I have an MQTT gateway with an ESP-NOW repeater. The MQTT gateway gets the accurate time and gives it to the repeater. However, the repeater will then end up giving its skewed time back to the MQTT gateway.

I was wondering if we should classify one of the neighbors as the "upstream" neighbor and one the "downstream" neighbor so that the time is only sent to the downstream neighbor and not back to the upstream neighbor. I think you said earlier that for now we should keep the setup as generic/flexible as possible.

[timmbogner]

I was wondering if we should classify one of the neighbors as the "upstream" neighbor and one the "downstream" neighbor so that the time is only sent to the downstream neighbor and not back to the upstream neighbor.

I've had a response draft I've been mulling over since yesterday that I'll post now before I head to work. There's actually another option I can get into later that you've brought up involving specific time-routing options.

Here is the scenario(s) I've been thinking about, let me know your thoughts. Am I overlooking some looping possibilities again?

# Gateway 0x01 boots and has WiFi enabled, so it checks to see if it can reach the time server every x minutes. If it gets a time successfully, it then sends it out to neighbors. Neighboring Gateway 0x02 gets that time, sets its own clock, and sends it on to its neighbors BUT NOT the one that it received it from (0x01). We actually may want do something similar with nodes, to allow for a situation where a node with a powered RTC feeds the time to the whole system. When a gateway re-sends the time that came from a node, it would need to temporarily remove that node from the ESP-NOW peer list when it shares the time with everyone else.

So essentially the device that knows that it has the correct time, be it by NTP or RTC, initiates a chain reaction that updates all the devices but never goes backwards.

OR

We keep all the time scheduling stuff, but each gateway now keeps track of where the time came from and sends it everywhere else, but never back to where it came from.

[timmbogner]

Other ideas: The gateway with the known time is the only one to initiate time updates, and it sends to everyone BUT ignores subsequent time updates for a certain period of time. So like, a gateway sends out the time, but then can't do it again for a minute, even if a time update comes in.

The idea I alluded to this morning: Basically assign where the time goes when it comes in, or when it is scheduled. It would essentially be an "outbound list".

[aviateur17]

I like the idea of the gateways/repeaters keep track of the source address of where the time packet was received and when sending it out to other notes checking to make sure that it does not send the time to that address.

[timmbogner]

I'm back to it, and first let me mention this is suuuper nifty and fun! Timekeeping is going to be such a powerful feature...

Currently I can't get any device to receive a time via LoRa. Pings work fine, but when the gateway sends a LoRa time packet, nothing happens on the next gateway or any nodes. My setup is an Ethernet gateway connected to a LoRa32 gateway via ESP-NOW, then to another LoRa32 via LoRa. I have tried with both a gateway and a node for the next device. Timekeeping works a charm up to the LoRa hop.

Any ideas?

[aviateur17]

I think there was a hook in LoRa that I forgot to implement. I think I have it almost ready to go but haven't submitted it yet. I'm trying to think whether it was in the gateway code, I believe. So gateways wouldn't be able to send to other gateways without the hook. I'll try to get that submitted tonight. Sorry about that.

[timmbogner]

No worries! I have been working to get my T-Watch going on the system. One thing I've changed for this is to have the gateway send the time to a controller immediately after registering. I'll push that commit later if it's alright with you, as well as adding the following sketch to the controller examples. It works, but in UTC. Can you point me in the right direction?

Here is the RTC library being used, I think setDateTime(uint16_t year, uint8_t month, uint8_t day, uint8_t hour, uint8_t minute, uint8_t second) may be the useful function.

//  FARM DATA RELAY SYSTEM
//
//  LilyGo T-Watch 2020 Example
//
//  Developed by Timm Bogner (timmbogner@gmail.com) in Urbana, Illinois, USA.
//

#define LILYGO_WATCH_2020_V1

#include <LilyGoWatch.h>
#include "fdrs_node_config.h"
#include <fdrs_node.h>

TTGOClass *ttgo;
TFT_eSPI *tft;
PCF8563_Class *rtc;

uint32_t interval = 0;

void fdrs_recv_cb(DataReading theData) {
  DBG("ID: " + String(theData.id));
  DBG("Type: " + String(theData.t));
  DBG("Data: " + String(theData.d));
}

void setup() {
  Serial.begin(115200);
  //Get watch instance
  ttgo = TTGOClass::getWatch();
  // Initialize the hardware
  ttgo->begin();
  // Turn on the backlight
  ttgo->openBL(); 
  // Receive as a local variable for easy writing
  rtc = ttgo->rtc;  
  tft = ttgo->tft;

  beginFDRS();
  pingFDRS(1000);
  addFDRS(fdrs_recv_cb);
  subscribeFDRS(READING_ID);
  delay(100);
  rtc->syncToRtc();
}

void loop() {
  loopFDRS();

  if (millis() - interval > 1000) {
    interval = millis();
    tft->setTextColor(TFT_BLUE, TFT_BLACK);
    tft->drawString(rtc->formatDateTime(PCF_TIMEFORMAT_MM_DD_YYYY), 50, 200, 4);
    tft->setTextColor(TFT_GREEN, TFT_BLACK);
    tft->drawString(rtc->formatDateTime(PCF_TIMEFORMAT_HMS), 5, 75, 7);
  }
}

[aviateur17]

No worries! I have been working to get my T-Watch going on the system. One thing I've changed for this is to have the gateway send the time to a controller immediately after registering. I'll push that commit later if it's alright with you

Sounds good!

, as well as adding the following sketch to the controller examples. It works, but in UTC. Can you point me in the right direction?

Here is the RTC library being used, I think setDateTime(uint16_t year, uint8_t month, uint8_t day, uint8_t hour, uint8_t minute, uint8_t second) may be the useful function.

syncToRTC() in your example will take the system time and set it in the watch RTC. I would wrap that call with an

if(validTime()) {
    rtc->syncToRtc();
}

syncToSystem() will read the time from the RTC and set it in the running program

rtc->syncToSystem();

[timmbogner]

Sorry, I should have been more specific with my issue. Everything in that sketch works, except the display shows the time in UTC, which is what syncToRtc() sets it as. Is there an easy way to set it to the date/time adjusted for timezone and DST?

[aviateur17]

Sorry, I should have been more specific with my issue. Everything in that sketch works, except the display shows the time in UTC, which is what syncToRtc() sets it as. Is there an easy way to set it to the date/time adjusted for timezone and DST?

The way I've coded the time stuff in FDRS is to store the time in UTC and then convert to local time based on the configured offset and whether the current time is in DST or STD time. The NTP information is in UTC and I think that's the most flexible way to store the time. So if you're seeing the time in UTC then it sounds like it's not being converted to your local time. The printTime() function is what takes the UTC time and converts the output to local time to be displayed. So I guess the question is whether that library expects local time to be stored in RTC or if it expects UTC. How does the library handle changeover from DST to STD time and vice versa? It's almost like it expects the RTC time to be in the local time zone. So in order to handle that you'd have to get the system time and then offset it by your time zone and then store it to the watch RTC. I guess what I would probably do is just replace the formatDateTime lines by your own function calls. So the class calls:

snprintf(format, sizeof(format), "%02d-%02d-%02d", t.month, t.day, t.year);

and

snprintf(format, sizeof(format), "%02d:%02d:%02d", t.hour, t.minute, t.second);

so I would substitute those calls with the strftime function call similar to the printTime() function. Syntax for strftime() can be found at the URL below: https://www.tutorialspoint.com/c_standard_library/c_function_strftime.htm

char strftime_buf[64];
time_t local = time(NULL) + (isDST?dstOffset:stdOffset);
localtime_r(&local, &timeinfo);
strftime(strftime_buf, sizeof(strftime_buf), "%m-%d-%Y", &timeinfo);  // generate MM-DD-YYYY local time character array
tft->setTextColor(TFT_BLUE, TFT_BLACK);
tft->drawString(strftime_buf, 50, 200, 4);
tft->setTextColor(TFT_GREEN, TFT_BLACK);
strftime(strftime_buf, sizeof(strftime_buf), "%I:%M:%S", &timeinfo); // generate HH:MM:SS local time character array
tft->drawString(strftime_buf, 5, 75, 7);

If there is a delay you can use two char arrays and generate both strings before displaying them.

I think something like that should work.

[timmbogner]

I think the RTC does expect it to be local time. I guess we ought to find out what the more "standard" way of operating is. It would make sense that a device such as a clock or watch would set to local time internally. Would a datalogger be similar or would it use UTC?

It seems like since you did all the work to make a nice DST routine, we should make the result a little more accessible to users. I have two ideas, let me know if either one seems good. One is to simply make the final offset (after DST adjustment) accessible through a function. The other is to just have two matching functions that return tm structs of either local or UTC.

What do you think? I may do option 2 either way. And we can also do neither or both.

[aviateur17]

I think the RTC does expect it to be local time. I guess we ought to find out what the more "standard" way of operating is. It would make sense that a device such as a clock or watch would set to local time internally. Would a datalogger be similar or would it use UTC?

I started out using local time as the RTC time but I think it's more universal to use the UTC time and just convert to whatever standard you want to use. I could see the datalogger data going either way. It really depends upon how "global" you want the data to be. After starting with local time I switched to UTC time because that is how it is utilized in NTP and I think the DST/STD conversion is way easier because you really do not have to fool with going backwards in time in the local time. I have no idea what the Arduino standard is but in the building controls and industrial controls world I think UTC is the standard. Also, pilots use UTC as the standard but they are traversing between different time zones all the time. I think an argument can be made either way but I think storing as UTC is better and calculating the offset to print the local time is the way to go.

It seems like since you did all the work to make a nice DST routine, we should make the result a little more accessible to users. I have two ideas, let me know if either one seems good. One is to simply make the final offset (after DST adjustment) accessible through a function. The other is to just have two matching functions that return tm structs of either local or UTC.

What do you think? I may do option 2 either way. And we can also do neither or both.

I guess I would prefer a function that outputs local time in whatever manner is desired given my preference to have the underlying time in UTC.

[timmbogner]

~Yeah I see why UTC RTC makes everything easier. No problem with keeping it how it is. I realized my "function that returns the offset" idea is a bit superfluous. It would be good enough to just know if user is in DST or not...~

~SO what if you essentially just made checkDST() return a boolean so the user can have access to it? Then the user can make the logic to decide which offset macro to use to get the local time.~

We could put a pin in the idea of functions that return tm structs for later. I'm not sure yet if that will be very complicated or useful.

In unrelated news that I find sort of funny: Why didn't Espressif come up with a more unique acronym for ESP-RTC??😄

[timmbogner]

OOPS, SORRY! I just noticed bool isDST; is global, and is p much what I described, right?

Yeah I totally misread your example. My bad.

[timmbogner]

I think ESP-NOW is pretty much solid. I can't get devices to receive a LoRa time, though. I've got Gateway 0x01 connected to MQTT and LoRa, then 0x03 is straight LoRa. 0x01 says it is sending, but I can't get the repeater 0x03 or a node to receive it. DataReadings and pings still make it okay.

Any ideas? I'm going to start working through it with DBGs to find where things are going wrong.

[aviateur17]

I think ESP-NOW is pretty much solid. I can't get devices to receive a LoRa time, though. I've got Gateway 0x01 connected to MQTT and LoRa, then 0x03 is straight LoRa. 0x01 says it is sending, but I can't get the repeater 0x03 or a node to receive it. DataReadings and pings still make it okay.

Any ideas? I'm going to start working through it with DBGs to find where things are going wrong.

Sorry, I've been neglecting this as I've been busy with other things. Also been troubleshooting issues and have limited time. I think LoRa is working on my code but I'm troubleshooting multiple instances of time being sent. I'll submit what I have and then troubleshoot later.

[aviateur17]

I think ESP-NOW is pretty much solid. I can't get devices to receive a LoRa time, though. I've got Gateway 0x01 connected to MQTT and LoRa, then 0x03 is straight LoRa. 0x01 says it is sending, but I can't get the repeater 0x03 or a node to receive it. DataReadings and pings still make it okay.

Any ideas? I'm going to start working through it with DBGs to find where things are going wrong.

As far as LoRa goes the commit from a few days ago should work. Are you using the latest commits in this branch? I'll send my output if I have time later on tonight but I'm not sure that I'll have time.

[aviateur17]

Here is my debug output from:

ESPNOW -> LoRa gateway - address 0x22 (time received from 0x21):

Local date/time is: Fri Jun  9 07:07:53 2023 DST
Local date/time is: Fri Jun  9 07:08:53 2023 DST
Local date/time is: Fri Jun  9 07:09:53 2023 DST
Local date/time is: Fri Jun  9 07:10:53 2023 DST
Local date/time is: Fri Jun  9 07:11:53 2023 DST
Incoming ESP-NOW System Packet from 0x21
Received time via ESP-NOW from 0x21
ESP-NOW time master is 0x21
Received time via ESP-NOW from 0x21
Time adjust 1 secs
ESP-NOW time master is 0x21
Sending out time
Sending time via LoRa
Sending time to LoRa broadcast
Time adjust 0 secs
Transmitting LoRa message of size 11 bytes with CRC 0xa222 to LoRa MAC 0xffff
Sending time to LoRa Neighbor 1
Transmitting LoRa message of size 11 bytes with CRC 0x37ba to LoRa MAC 0xee23
Sending out time
Sending time via ESP-NOW
Sending time to ESP-NOW Peer 1
Sending time to ESP-NOW registered peers
Sending time via LoRa
LoRa airtime: 366782ms
Sending time to LoRa broadcast
Transmitting LoRa message of size 11 bytes with CRC 0xa222 to LoRa MAC 0xffff
Sending time to LoRa Neighbor 1
LoRa airtime: 366907ms
Transmitting LoRa message of size 11 bytes with CRC 0x37ba to LoRa MAC 0xee23
Sending time via ESP-NOW
LoRa airtime: 366987ms
Sending time to ESP-NOW Peer 1
Sending time to ESP-NOW registered peers
Incoming LoRa. Size: 11 Bytes, RSSI: -62.00dBm, SNR: 9.50dB, PacketCRC: 0x6db8, Total LoRa received: 0
We have received a ping request from 0xee23, Replying.
Transmitting LoRa message of size 11 bytes with CRC 0xfd98 to LoRa MAC 0xee23
LoRa airtime: 367248ms

LoRa -> LoRa Gateway - address 0x23 (time received from 0x22):

Incoming LoRa. Size: 11 Bytes, RSSI: -64.00dBm, SNR: 9.75dB, PacketCRC: 0x37ba, Total LoRa received: 0
Time adjust 1686312466 secs
Time change from STD -> DST
Sending out time
Sending time via LoRa
Transmitting LoRa message of size 11 bytes with CRC 0x6232 to LoRa MAC 0xffff
Time rcv from LoRa 0xee22
Transmitting LoRa message of size 11 bytes with CRC 0x6db8 to LoRa MAC 0xee22
LoRa ping sent to address: 0xee22
LoRa airtime: 255177ms
Incoming LoRa. Size: 11 Bytes, RSSI: -65.00dBm, SNR: 9.50dB, PacketCRC: 0xfd98, Total LoRa received: 1, CRC Ok Pct 100.00%
We have received a ping reply via LoRa from address ee22
LoRa Ping Returned: 295ms.
Time adj by 0 secs
Local date/time is: Fri Jun  9 07:12:45 2023 DST
Local date/time is: Fri Jun  9 07:13:45 2023 DST

[timmbogner]

Well please don't feel like you need to do anything in a hurry. I'm also a bit preoccupied, so nothing is time-critical. No pun intended.

In fact, if you do get some free time you should check out this series about the radio systems on the Voyager probes from EEVblog!

Just saw your latest post: I will try the ESP-NOW to LoRa time-hop again to see what my results are.

[timmbogner]

I've got it working on a different board. For some reason the TTGO LoRa32 v1 has an issue where it fully appears to transmit the time, but doesn't actually send anything. It's now running okay on the T3S3. The T3S3 seemed like it had issues at first, but dropped the issues and started working after some troubleshooting and I'm unclear what fixed them.

[timmbogner]

Hey Jeff, I am continuing to have issues with gateways not sending LoRa time. I can confirm that the same problem occurring on the LoRa32 is also occurring with an ESP8266 and SX1278 module.

Using T7-S3 and SX1278 it seems to send only one of the three LoRa time packets. This is similar to what the T3-S3 (with SX1262) did at first, though I'm not sure how I corrected it. I think this could be an issue with timing, and several receptions/transmissions happening too quickly. The packet that it does receive is the one destined for neighbor#2, which is the last packet broadcasted.

I think these are two separate issues, but I'm not positive. Could you share the the setup(s) that you have working? I have done some troubleshooting and cannot solve the first issue.

EDIT: I think I've solved the second issue: In sendTimeLoRa(), each LoRa packet is sent to the radio without waiting for the previous to finish. I think the solution to this will be a similar setup to the asynchronous LoRa release function.

[aviateur17]

Hey Jeff, I am continuing to have issues with gateways not sending LoRa time. I can confirm that the same problem occurring on the LoRa32 is also occurring with an ESP8266 and SX1278 module.

Using T7-S3 and SX1278 it seems to send only one of the three LoRa time packets. This is similar to what the T3-S3 (with SX1262) did at first, though I'm not sure how I corrected it. I think this could be an issue with timing, and several receptions/transmissions happening too quickly. The packet that it does receive is the one destined for neighbor#2, which is the last packet broadcasted.

I've got a setup where I have a LoRa Neighbor 1 (N1) and no neighbor 2 and no LoRa sensor yet. The N1 node doesn't reliably receive LoRa time periodically from the LoRa repeater that should be sending out the time. I'm not sure whether the issue is the transmitting repeater side or the receiving neighbor side yet. I'll have to do more testing on this. I'll add in some LoRa pings and add a LoRa sensor to determine what is happening.

I think these are two separate issues, but I'm not positive. Could you share the the setup(s) that you have working? I have done some troubleshooting and cannot solve the first issue.

All of my LoRa nodes are using a Wemos ESP32 with the Hallard LoRa shield.

EDIT: I think I've solved the second issue: In sendTimeLoRa(), each LoRa packet is sent to the radio without waiting for the previous to finish. I think the solution to this will be a similar setup to the asynchronous LoRa release function.

Thanks, I'll look for this and see if this is part of the problem.

[aviateur17]

@timmbogner, just wanted to let you know I'm continuing to work on this feature. I don't know for sure when I'll have the whole thing done but I'm hoping to be able to spend several hours a week on it. I'm not really sure where I left off but I have time going from MQTT node -> Serial -> ESP-NOW -> LoRa -> LoRa - all of those gateways so I need to test the sensor and gateway code. Anyways, just wanted to let you know.

[timmbogner]

Greetings @aviateur17, that sounds awesome!

If I recall correctly, the ESP-NOW implementation was perfect, but there was a glitch in the LoRa portion. One thing I'd propose is that we start with a fresh pull request. This one is kind of a lot to go through, although mostly due to my own blunders.

An important upcoming change is going to be functionally major. I'm going to extend the 'data' portion of the DataReading structure to 5 bytes, and use CBOR to allow the data to be boolean, integer, binary, or float. This will only extend the DataReading to 8 bytes, which I think is pretty reasonable. There shouldn't be too many external changes for users. I will add new iterations of loadFDRS() for the different datatypes and then everything else will be handled internally. What do you think? SystemPackets will stay the same, so I'm not sure if it will affect your work here on timekeeping.

I have a rough sketch worked out with a lot of the experimentation done using this TinyCBOR library. I'll make it a priority to make some progress on it and post something today or tomorrow.

Thanks!

[aviateur17]

@timmbogner , agreed on posting a new pull request. When I'm ready or close to ready and done with testing I'll post a new PR against the updated main branch.

Regarding the change in the structure of the DataReadings, in the LoRa code I'm using the length to differentiate between valid packets and invalid packets and then going in and dissecting what is actual data and what are the headers so once you have that change we'll need to go into that LoRa code and modify those parts. Shouldn't be too difficult unless the size of the data is variable in nature.

[timmbogner]

Shouldn't be too difficult unless the size of the data is variable in nature.

Cool! Nope, the data will now be a constant 5 bytes, so it'll look like this (probably):

typedef struct __attribute__((packed)) DataReading {
  uint8_t[5] d;
  uint16_t id;
  uint8_t t;

} DataReading;

I'm relying on the fact that (in most cases) sizeof(DataReading) was used in place of the actual number 7, so I think a lot of your code should be good to go. I'm sure there will be some kind of issue though.

[aviateur17]

Closing old PR request. See PR #193