search

SuperHouse / esp-open-rtos

Open source FreeRTOS-based ESP8266 software framework
BSD 3-Clause "New" or "Revised" License
1.54k stars 490 forks source link

SNTP support #108

Closed doragasu closed 8 years ago

doragasu commented 8 years ago

I'm trying to add SNTP support, using lwip SNTP contribution. Right now I have some things working:

  1. lwip SNTP builds and works perfect using raw API. I can get the number of seconds and microseconds from Epoch, and values are correct.
  2. Time conversion functions from newlib somehow work: If I call ctime() passing the number of seconds obtained from SNTP, I get a properly formatted string with the UTC time.

But there are some things that are not working and I would like to implement:

  1. Currently date/time once obtained from SNTP, is not used to adjust the system time. I would like to do it.
  2. I'm cannot get time functions to use time zones and daylight saving options. Manually setting _localtime and _daylight vars from time.h does not affect the output of ctime(), UTC time is always shown.

To implement (1), there are several possible routes:

  1. Using RTC timer. Has the problem that this timer is 32-bit and wraps approximately each 6 hours. So an interrupt should be used to increment an auxliliar variable each time this timer overflows. As I cannot find information on RTC interrupts, I don't know how to do this.
  2. Using RTOS tick (as returned by xTaskGetTickCount()). Has the same problem as (1), and even worse, as this 32-bit value overflows sooner. I also suspect this cannot generate a timer on overflow that I can use.
  3. Using FRC2. Has the same problem as (1). Also it is supposed to be used by FreeRTOS, but I have been experimenting using and even stopping this timer and FreeRTOS tasks continue running apparently unaffected. I don't know if this timer can generate an interrupt on overflow, but I suppose setting it to generate a compare with 0 interrupt, will cause the same outcome.
  4. Using FRC1. Unfortunately it prevents you from using it for other useful tasks (like PWM).
  5. Not using timer interrupts at all. Tim Gremalm suggested on IRC that I could e.g. use the RTC timer, and make sure I check for overflows each time someone wants to get the time, and also each time SNTP updates the time, being the SNTP update interval lower than 6 hours. It has the problem that if network is down for more than 6 hours, errors might occur.

Any suggestions about the best way to tackle this?

Also to solve the problem with time zones and daylight savings, I could implement a simple solution that just adds/subtracts multiples of 3600 to time references, but I'm asking here just in case there is a more elegant solution.

doragasu commented 8 years ago

I have just tried writing to RTC.COUNTER, and it has no effect! Maybe this register is read only, or requires a magic unlock sequence before writing.

projectgus commented 8 years ago

That's annoying. It would be good to fully decipher the possible RTC functions.

Maybe a simple approach could be to use the FreeRTOS timer framework. You use xTaskGetTickCount() to set your reference tick count, and then set up a FreeRTOS timer with xTimerCreate(). Set a maximum period of something like freeRTOSTickCount / 2. You need some safety margin because FreeRTOS timers run at low task priority. When the timer times out, update the time base and set a new timer for freeRTOSTickCount / 2, again.

You could double up the timer functionality to also trigger a new clock sync via SNTP.

Or alternatively if you want periodic SNTP syncing then you could put the whole thing in a task, where the task behaviour is like this:

A dedicated task would use more resources than a simple timer, but it uses them in a predictable way.

doragasu commented 8 years ago

First attempt, using RTC and checking for timer wraps each time date/time is requested. I have also avoided using floating point for computations. It looks like it's working right now, but I haven't yet tested if the timer wrap detection/compensation code works as intended (I need a lenghty run to see the wrap in action). There is a lot of debug code that can be removed:

#include <stdio.h>
#include <espressif/esp_common.h>
#include <esp/timer.h>
#include <esp/rtc_regs.h>
#include "sntp.h"

// FreeRTOS headers needed only by debug code
#include "FreeRTOS.h"
#include "task.h"

#define TIMER_COUNT         RTC.COUNTER

// TODO: Try moving these global variables to RTC scratch registers (if writable!)
static uint64_t sntp_base;
static uint32_t tim_ref;
// Note value is fixed point (Q20.12)
static uint32_t cal;

// Initialization of SNTP layer from lwIP
void sntp_init(void);

void sntp_initialize(void) {
    sntp_base = 0;
    tim_ref = TIMER_COUNT;
    cal = 1;    // To avoid div by 0 exceptions if requesting time before first SNTP update
    sntp_init();
}

// Check if a timer wrap has occurred. Compensate sntp_base reference
// if affirmative.
// TODO: think about multitasking and race conditions
static inline void sntp_check_timer_wrap(uint32_t current_value) {
    if (current_value < tim_ref) {
        // Timer wrap has occurred, compensate by subtracting 2^32 to ref.
        sntp_base -= 1LLU<<32;
        // DEBUG
        printf("\nTIMER WRAPPED!\n");
    }
}

// Return secs. If us is not a null pointer, fill it with usecs
time_t sntp_get_rtc_time(int32_t *us) {
    time_t secs;
    uint32_t tim;
    uint64_t base;

    tim = TIMER_COUNT;
    // Check for timer wrap
    sntp_check_timer_wrap(tim);
    base = sntp_base + tim - tim_ref;
    secs = base * cal / (1000000U<<12);
    if (us) {
        *us = base * cal % (1000000U<<12);
    }
    return secs;
}

/// Update RTC timer. Called by SNTP module each time it receives an update.
void sntp_update_rtc(time_t t, uint32_t us) {
    // DEBUG: Compute and print drift
    int64_t sntp_current = sntp_base + TIMER_COUNT - tim_ref;
    int64_t sntp_correct = (((uint64_t)us + (uint64_t)t * 1000000U)<<12) / cal;
    printf("\nRTC Adjust: drift = %ld ticks, cal = %d\n", (time_t)(sntp_correct - sntp_current), cal);

    tim_ref = TIMER_COUNT;
    cal = sdk_system_rtc_clock_cali_proc();
    sntp_check_timer_wrap(tim_ref);

    sntp_base = (((uint64_t)us + (uint64_t)t * 1000000U)<<12) / cal;

    // DEBUG: Print obtained secs and check calculated secs are the same
    time_t deb = sntp_base * cal / (1000000U<<12);
    printf("\nT: %lu, %lu, %s\n", t, deb, ctime(&deb));
    // DEBUG: Sleep 6 seconds and check time got increased as expected
    vTaskDelay(5000 / portTICK_RATE_MS);
    deb = (sntp_base + TIMER_COUNT - tim_ref) * cal / (1000000U<<12);
    printf("\nT5: %lu, %lu, %s\n", t, deb, ctime(&deb));
}

This is the output of a typical run (using also a task that requests date/time each 5s, and with the SNTP task requesting an update each minute):

dhcp client start...
ip:192.168.0.120,mask:255.255.255.0,gw:192.168.0.1
Starting SNTP... DONE!

RTC Adjust: drift = 858559661 ticks, cal = 1

T: 1457007906, 1457007906, Thu Mar  3 12:25:06 2016

TIME: Thu Mar  3 12:25:11 2016

T5: 1457007906, 1457007911, Thu Mar  3 12:25:11 2016

TIME: Thu Mar  3 12:25:16 2016
TIME: Thu Mar  3 12:25:21 2016
TIME: Thu Mar  3 12:25:26 2016
TIME: Thu Mar  3 12:25:31 2016
TIME: Thu Mar  3 12:25:36 2016
TIME: Thu Mar  3 12:25:41 2016
TIME: Thu Mar  3 12:25:46 2016
TIME: Thu Mar  3 12:25:51 2016
TIME: Thu Mar  3 12:25:56 2016
TIME: Thu Mar  3 12:26:01 2016
TIME: Thu Mar  3 12:26:06 2016
TIME: Thu Mar  3 12:26:11 2016

RTC Adjust: drift = 10301 ticks, cal = 25041

T: 1457007971, 1457007971, Thu Mar  3 12:26:11 2016

TIME: Thu Mar  3 12:26:16 2016

T5: 1457007971, 1457007976, Thu Mar  3 12:26:16 2016

TIME: Thu Mar  3 12:26:21 2016
TIME: Thu Mar  3 12:26:26 2016
TIME: Thu Mar  3 12:26:31 2016
TIME: Thu Mar  3 12:26:36 2016
TIME: Thu Mar  3 12:26:41 2016
TIME: Thu Mar  3 12:26:46 2016
TIME: Thu Mar  3 12:26:51 2016
TIME: Thu Mar  3 12:26:56 2016
TIME: Thu Mar  3 12:27:01 2016
TIME: Thu Mar  3 12:27:06 2016
TIME: Thu Mar  3 12:27:11 2016
TIME: Thu Mar  3 12:27:16 2016

RTC Adjust: drift = -5775 ticks, cal = 25081

To get the drift in seconds, you must use the cal value: drift_seconds = ticks * cal / (2^12) / 1e6 E.g. for the -5775 ticks avobe it would be: drift_seconds = -5775 * 25081 / (2^12) / 1e6 = -0.035 s

foogod commented 8 years ago

This seems like a pretty good approach, at least for the time being..

Unfortunately, the RTC counter is based off of an internal RC oscillator instead of the main CPU crystal, which means it will vary substantially more with temperature than most of the other methods of tracking system time, so you'll get a lot more drift. One of the things on my own "want to do" list is eventually to create some system routines to maintain/report a high-precision monotonic system clock by querying CCOUNT (which could be used as a much more accurate base for this sort of thing), but unfortunately that's affected by the CPU clock frequency setting, which means before that can be done we need to build some hooks into the system routines so we can know when things switch the clock rate back and forth (and compensate accordingly), which basically means we need our own implementation of sdk_system_update_cpu_freq first..

(I actually have a reverse-engineered copy of that routine already as part of my efforts to RE all of user_interface.c, but haven't finished with some of the other routines in that file.. I suppose it's possible I could just pull that routine out independently for now so we could start hacking on it.. I'll have to look at how feasible that is..)

On Thu, Mar 3, 2016 at 4:31 AM, doragasu notifications@github.com wrote:

First attempt, using RTC and checking for timer wraps each time date/time is requested. I have also avoided using floating point for computations. It looks like it's working right now, but I haven't yet tested if the timer wrap detection/compensation code works as intended (I need a lenghty run to see the wrap in action). There is a lot of debug code that can be removed:

include

include <espressif/esp_common.h>

include <esp/timer.h>

include <esp/rtc_regs.h>

include "sntp.h"

// FreeRTOS headers needed only by debug code

include "FreeRTOS.h"

include "task.h"

define TIMER_COUNT RTC.COUNTER

// TODO: Try moving these global variables to RTC scratch registers (if writable!)static uint64_t sntp_base;static uint32_t tim_ref;// Note value is fixed point (Q20.12)static uint32_t cal; // Initialization of SNTP layer from lwIPvoid sntp_init(void); void sntp_initialize(void) { sntp_base = 0; tim_ref = TIMER_COUNT; cal = 1; // To avoid div by 0 exceptions if requesting time before first SNTP update sntp_init(); } // Check if a timer wrap has occurred. Compensate sntp_base reference// if affirmative.// TODO: think about multitasking and race conditionsstatic inline void sntp_check_timer_wrap(uint32_t current_value) { if (current_value < tim_ref) { // Timer wrap has occurred, compensate by subtracting 2^32 to ref. sntp_base -= 1LLU<<32; // DEBUG printf("\nTIMER WRAPPED!\n"); } } // Return secs. If us is not a null pointer, fill it with usecstime_t sntp_get_rtc_time(int32_t *us) { time_t secs; uint32_t tim; uint64_t base;

tim = TIMER_COUNT;
// Check for timer wrap
sntp_check_timer_wrap(tim);
base = sntp_base + tim - tim_ref;
secs = base * cal / (1000000U<<12);
if (us) {
    *us = base * cal % (1000000U<<12);
}
return secs;

} /// Update RTC timer. Called by SNTP module each time it receives an update.void sntp_update_rtc(time_t t, uint32_t us) { // DEBUG: Compute and print drift int64_t sntp_current = sntp_base + TIMER_COUNT - tim_ref; int64_t sntp_correct = (((uint64_t)us + (uint64_t)t * 1000000U)<<12) / cal; printf("\nRTC Adjust: drift = %ld ticks, cal = %d\n", (time_t)(sntp_correct - sntp_current), cal);

tim_ref = TIMER_COUNT;
cal = sdk_system_rtc_clock_cali_proc();
sntp_check_timer_wrap(tim_ref);

sntp_base = (((uint64_t)us + (uint64_t)t * 1000000U)<<12) / cal;

// DEBUG: Print obtained secs and check calculated secs are the same
time_t deb = sntp_base * cal / (1000000U<<12);
printf("\nT: %lu, %lu, %s\n", t, deb, ctime(&deb));
// DEBUG: Sleep 6 seconds and check time got increased as expected
vTaskDelay(5000 / portTICK_RATE_MS);
deb = (sntp_base + TIMER_COUNT - tim_ref) * cal / (1000000U<<12);
printf("\nT5: %lu, %lu, %s\n", t, deb, ctime(&deb));

}

This is the output of a typical run (using also a task that requests date/time each 5s, and with the SNTP task requesting an update each minute):

dhcp client start... ip:192.168.0.120,mask:255.255.255.0,gw:192.168.0.1 Starting SNTP... DONE!

RTC Adjust: drift = 858559661 ticks, cal = 1

T: 1457007906, 1457007906, Thu Mar 3 12:25:06 2016

TIME: Thu Mar 3 12:25:11 2016

T5: 1457007906, 1457007911, Thu Mar 3 12:25:11 2016

TIME: Thu Mar 3 12:25:16 2016 TIME: Thu Mar 3 12:25:21 2016 TIME: Thu Mar 3 12:25:26 2016 TIME: Thu Mar 3 12:25:31 2016 TIME: Thu Mar 3 12:25:36 2016 TIME: Thu Mar 3 12:25:41 2016 TIME: Thu Mar 3 12:25:46 2016 TIME: Thu Mar 3 12:25:51 2016 TIME: Thu Mar 3 12:25:56 2016 TIME: Thu Mar 3 12:26:01 2016 TIME: Thu Mar 3 12:26:06 2016 TIME: Thu Mar 3 12:26:11 2016

RTC Adjust: drift = 10301 ticks, cal = 25041

T: 1457007971, 1457007971, Thu Mar 3 12:26:11 2016

TIME: Thu Mar 3 12:26:16 2016

T5: 1457007971, 1457007976, Thu Mar 3 12:26:16 2016

TIME: Thu Mar 3 12:26:21 2016 TIME: Thu Mar 3 12:26:26 2016 TIME: Thu Mar 3 12:26:31 2016 TIME: Thu Mar 3 12:26:36 2016 TIME: Thu Mar 3 12:26:41 2016 TIME: Thu Mar 3 12:26:46 2016 TIME: Thu Mar 3 12:26:51 2016 TIME: Thu Mar 3 12:26:56 2016 TIME: Thu Mar 3 12:27:01 2016 TIME: Thu Mar 3 12:27:06 2016 TIME: Thu Mar 3 12:27:11 2016 TIME: Thu Mar 3 12:27:16 2016

RTC Adjust: drift = -5775 ticks, cal = 25081

To get the drift in seconds, you must use the cal value: drift_seconds = ticks * cal / (2^12) / 1e6 E.g. for the -5775 ticks avobe it would be: drift_seconds = -5775 * 25081 / (2^12) / 1e6 = 0.035 s.

— Reply to this email directly or view it on GitHub https://github.com/SuperHouse/esp-open-rtos/issues/108#issuecomment-191742558 .

doragasu commented 8 years ago

Although far from perfect, RC oscillator is precise enough for me, and is the way to go if you want the date/time to survive waking up from deep sleep. But I think that it would be easy to support simultaneously several mechanisms for keeping the time, letting the user choosing between the "innacurate" RTC, or other more accurate timer.

One interesting thing about the RC timer, is that it is continuously being callibrated. The sdk_system_rtc_clock_cali_proc() function returns the number of RTC ticks per microsecond. I don't know how this number is computed by the SDK code, but it's continuously changing. Maybe it depends on some kind of internal temperature reading, or is based on a measure of the difference between two timers, who knows. I read and apply this callibration value each time the SNTP routine adjusts the clock.

I have added a bit more code:

  1. Global variables are now saved into RTC scratch registers, so they should also survive a deep sleep (although I have not tested it).
  2. I have added a lazy support for timezone and daylight saving configuration. It's rudimentary but it looks like it's working.

In case anyone is interested in adding this to esp-open-rtos, I was wondering what's the best way to merge it, as an example? as an extra module? inside core?

foogod commented 8 years ago

It's actually not "continuously" being calibrated. Basically all sdk_system_rtc_clock_cali_proc does is it measures a couple of RTC ticks and then calculates how long (in processor cycles) it took for that to happen, and returns the result. It doesn't adjust anything in hardware, and only samples the tick ratio at that one particular instant in time, so it may not represent drifts that occur between sampling intervals.

Still, as you say, it's probably good enough for now, particularly if you are continually updating things via SNTP. (It probably wouldn't be good enough for a wall clock if you weren't calibrating it regularly from another source, though)

My plan with the HPMC was actually to have it save its state when going into deep sleep and then restore itself based on that and the RTC when waking up again, so if/when I ever implement that it should really be the best of both worlds, but as I said there are some prerequisites that may or may not take a while to happen first, so definitely not something you should wait around for..

In terms of integrating into esp-open-rtos, I'd say that your basic clock/timezone stuff sounds like something that would be good to try to build into the core, and possibly put the SNTP part into "extras".. If we are going to put the clock stuff in core, though, we should probably look into whether there is any RTC scratch space in the "system" area that isn't currently being used by the SDK stuff that it could use (the "user" area should ideally be left free for apps to do with as they please without worrying about stepping on core OS stuff). I remember doing some research into what parts of that space are actually used by the Espressif routines, but I don't remember what the results were off hand.. I'll have to go dig up my notes.

projectgus commented 8 years ago

This looks great! I'd be very happy to add whatever hooks or other functionality is needed in 'core' to support generic timebase/time synchronisation stuff.

One note, you might be on top of this already but your most solid drift calibration source is SNTP! As each callback fires, you know X RTC ticks == Y seconds of SNTP wall clock time so you can use that as a calibration base (unless I'm missing something). The only skew left is any latency in the ESP's network/task layers.

projectgus commented 8 years ago

Minor note I saw mentioned on the esp8266/Arduino gitter channel just now, apparently the RTC counter registers zero out on a wake-from-sleep reset. So you need to preserve the "trigger" value in the RTC RAM somewhere. Possibly the binary SDK layer already does this for us, but thought I'd write it down here as relevant.

yesco commented 8 years ago

This is good stuff.

One concern, which may not be achievable is if the time source apps observation can be made monotonically increasing. If it at each sync could jump back in time that's very important to take into account for any logger app that timestamps data. This, however, may be outside the scope here. Also, not sure how it would be documented. I guess in posix in general there may be no such guarantees, however, most computers don't continually update the time. On Mar 4, 2016 4:52 AM, "Alex Stewart" notifications@github.com wrote:

It's actually not "continuously" being calibrated. Basically all sdk_system_rtc_clock_cali_proc does is it measures a couple of RTC ticks and then calculates how long (in processor cycles) it took for that to happen, and returns the result. It doesn't adjust anything in hardware, and only samples the tick ratio at that one particular instant in time, so it may not represent drifts that occur between sampling intervals.

Still, as you say, it's probably good enough for now, particularly if you are continually updating things via SNTP. (It probably wouldn't be good enough for a wall clock if you weren't calibrating it regularly from another source, though)

My plan with the HPMC was actually to have it save its state when going into deep sleep and then restore itself based on that and the RTC when waking up again, so if/when I ever implement that it should really be the best of both worlds, but as I said there are some prerequisites that may or may not take a while to happen first, so definitely not something you should wait around for..

In terms of integrating into esp-open-rtos, I'd say that your basic clock/timezone stuff sounds like something that would be good to try to build into the core, and possibly put the SNTP part into "extras".. If we are going to put the clock stuff in core, though, we should probably look into whether there is any RTC scratch space in the "system" area that isn't currently being used by the SDK stuff that it could use (the "user" area should ideally be left free for apps to do with as they please without worrying about stepping on core OS stuff). I remember doing some research into what parts of that space are actually used by the Espressif routines, but I don't remember what the results were off hand.. I'll have to go dig up my notes.

— Reply to this email directly or view it on GitHub https://github.com/SuperHouse/esp-open-rtos/issues/108#issuecomment-191981385 .

doragasu commented 8 years ago

@foogod Yeah, I didn't express myself clear enough, I know RTC is not being continuously callibrated. If you look to the code I posted before, I read the cal value each time RTC is adjusted, and use the read value to "manually" adjust the time.

@projectgus I didn't think about using SNTP readings to calibrate RTC. Might be worth a try, but I wonder if it will be precise enough if doing reads with relative short invervals (e.g. minutes), as SNTP is not as robust as NTP, and I think it might be affected by network latencies.

About merging this stuff on esp-open-rtos, there's something I have not decided how to do. I was trying if possible not to modify sntp.c from lwip contribs. But sntp.c includes sntp.h and expects some stuff (like NTP servers, update delay, etc.) to be configured there using #defines. I don't think using tricks like #include_next is the way to go, so I'm considering getting my hands dirty and modifying sntp.c to add the functions needed to dynamically change this stuff.

In case anybody is interested in giving it a try and doing suggestions, I have uploaded the code here

foogod commented 8 years ago

So FYI, I have updated http://esp8266-re.foogod.com/wiki/Memory_Accesses_(IoT_RTOS_SDK_0.9.9)#rtcmem with the best info I have on what is used by the SDK in the "system RTC memory" region. It looks like (as best I can tell) the SDK uses the first 32 bytes (8 words) for saving/loading its own network-config stuff across deep-sleeps, and it apparently uses the top three words (61, 62, and 63) elsewhere for other things (not sure what).. As far as I can tell, though, there appears to be quite a bit in the middle which we can probably use for our own stuff..

doragasu commented 8 years ago

I'm wondering who the hell decided that it's a good idea to zero the RTC when waking from deep sleep. It doesn't make sense, since RTC continues running to wake the module. This will complicate preserving time when waking from deep sleep, to the point that it is impossible if waking from external events!

Also I'm wondering if all RTC registers are zeroed, or only the counter is lost. Weird weird weird stuff.

doragasu commented 8 years ago

I have implemented the interfaces to be able to dynamically change SNTP servers and update delay. I'm going to try merging the code with my fork of esp-open-rtos, to send a pull request when ready. Unless anyone suggests otherwise, I'm going to put my code on core and I'll also add an example on how to use it.

The only thing I'm not sure where to put, is the code I grabbed from lwIP contribs. Although I slightly modified it, I don't know if I should add it to core, to a subfolder inside lwip, or to other place.

As always, suggestions are welcome :-)

projectgus commented 8 years ago

It doesn't make sense, since RTC continues running to wake the module. This will complicate preserving time when waking from deep sleep, to the point that it is impossible if waking from external events!

Yes, it is odd. I think it's fairly easy to work around though, we just need to save any state that is needed (including a flag that we're going to sleep, and the expected timer wakeup value when the ESP will trigger the RESET pin).

Unless anyone suggests otherwise, I'm going to put my code on core and I'll also add an example on how to use it.

I agree anything to do with RTC management should go into core. However, I don't want to add a dependency on the lwip SNTP functions to core. We should be able to have hooks/functions in core to allow RTC management to work, without tying it to a particular implementation.

I don't want to make things too painful for you though, feel free to send a PR and we can go from there together.

The only thing I'm not sure where to put, is the code I grabbed from lwIP contribs. Although I slightly modified it, I don't know if I should add it to core, to a subfolder inside lwip, or to other place.

I think at the moment "extras". There's a discussion currently going about rearranging extras, and maybe creating a "network" directory with network support components. But, for now, "extras".

foogod commented 8 years ago

Yes, it is odd. I think it's fairly easy to work around though, we just need to save any state that is needed (including a flag that we're going to sleep, and the expected timer wakeup value when the ESP will trigger the RESET pin).

Unfortunately, that will only work for timed sleeps, not for anything that's designed to be woken up by an external signal. I hadn't realized that the RTC counter gets reset on waking up from sleep, that does sound really stupid. It's possible that's something somewhere in the SDK libs that we can fix (I hope).. for now this sort of workaround may be the best we can do..

doragasu commented 8 years ago

OK, PR sent. I have finally added everything (except the example) to extras. I suppose this issue can be closed unless you want to leave it open to discuss other SNTP/RTC details non related to the PR I have sent.

projectgus commented 8 years ago

Thanks for that. I think closing this in favour of #114 makes sense for now. Can reopen if there are further non-PR-related discussions.