ntp sync with ESP8266 ESP-12F

[venustrg]

got my STC15W408AS, added NTP sync with ESP8266 ESP-12F board. works fine for me, syncs not only hour/min/sec but all data including year, month, day, weekday. configured to sync every 8 hours by default. added short howto with pictures, see docs/ntp.

[zerog2k]

First off, let me say, this is really awesome!

I'm not sure if you saw it, but i had done some early work with similar idea (to parse/read/set time from GPS NMEA strings): https://github.com/zerog2k/stc_diyclock/blob/gps/src/uart.h https://github.com/zerog2k/stc_diyclock/blob/gps/src/gps.h (however this is now somewhat out of date with the rest of the clock code, but idea is there)

So two concerns/ideas:

1. looks like we are trying to do stuff (other than capture the serial input, buffer manipulation, and setting state/flags) in the uart servicing ISR. Can the setting of time be moved outside of ISR? (I did this by using a state variable to keep track of this, and service a "full" timestring buffer in the main loop, but maybe could be through simple flags, etc)
2. I wonder if we can combine into a format which would work to set time/date from either of ntp source (esp8266, etc) or gps nmea string input? Could we either identify and use either string type in the clock, or perhaps have the ntp/esp8266 output nmea format (so clock code only needs to recognize/use one type)?

Looks like there is already at least one esp8266 project out there which is does gps-emulation (uses ntp and outputs gps nmea strings): https://github.com/xSnowHeadx/FakeGPS/blob/master/fake_GPS.ino#L80 (this one has the advantage of using WifiManager to provide web configuration portal via AP for setup, etc - without needing to hardcode or set this up via terminal, etc)

What do you think?

[venustrg]

of course, i saw your ntp branch and got the uart1_init from it. I don’t have a real GPS module for testing, but I’ll try to redo my esp8266 so that it uses the nmea time transfer format. as I can see there is no day of the week in nmea, so i have to either calculate it or leave it with manual setting.

[venustrg]

please leave this pull request opened for a while, i will create a new one.

[zerog2k]

Ok interesting. Day of week (DOW) setting is not something I had considered about NMEA/GPS. Additionally I'm also thinking of how to generally handle the case of timezones.

In the case of normal GPS modules, the time will be in UTC - so in absence of other tricks, the clock would need some sort of UTC Time Zone offset setting/menu, e.g. UTC+0200 or UTC-0500. (This would need to be manually set by the operator, and adjusted for any Daylight savings changes.)

In the case of the fake_GPS.ino example I mentioned above, it tries to fix both (TZ/DST) offsets this by adjusting this in the GPRMC sentenence it outputs (although it makes for technically incorrect gps nmea output)

Yet another possibility: Maybe your esp8266 nodemcu lua script can send an additional custom "sentence" which includes day-of-week and TZ offsets. This sentence need not be any NMEA compliant sentence. This clock code could recognize this special sentence and apply the settings/corrections (any other NMEA parsers would just ignore it as invalid). (So "fake" gps would send valid GPRMC or similar for setting time/date, and a custom sentence which clock recognizes for setting TZ/DOW).

So for this clock, in the case where we have valid, real GPS NMEA input, we can set the date & time automatically (but DOW and TZ would need to be set manually by operator), and in the case where we are using fake GPS from ESP8266, both of these can be set automatically.

Yet another issue I recall having with my early GPS code was it was trying to set the time too frequently, as GPRMC sentences come in every second or two when it has lock. While this can be controlled easily in fake gps, I was thinking of having some way to only apply incoming updates if some time has elapsed since previous good update has been applied, e.g. 1 hour, 1 day, etc.

Ideally, one could just compile the -DHAVE_GPS or similar option, and it would be flexible to work with either kind of GPS module (fake or real).

What do you think? I can work on some of this as well, if you want to split up the work - I just want to agree on what level of support and features this would have.

[venustrg]

my clock already uses nmea to transmit time with crc generating and checking at standard gps gpeed 9600. day of week set automatically. it works but right now my nodemcy apply tz offset before transmitting. i think there are no much chances clock will migrate from one zone to another, so it just can be compiled with WITH_NMEA, WITH_NMEA_TZ=XX and WITH_NMEA_DS. there is no space to make these values adjustable.

i think i will try do not set time directly from nmea but convert it to unix timestamp, which can be easily adjusted for local TZ offset and also can be corrected e.g. for -1 sec to compensate transfer etc. delays. just need time to write to_unix()/from_unix() conversions.

[venustrg]

can't find any mktime()/localtime() implementation to fit in memory. even with --stack-auto. TZ/DST without unixtime is tricky.

[venustrg]

6111 bytes with auto DOW but without TZ handling. i still searching. it's easy on ESP8266 but not on 8051

[zerog2k]

Yeah i would think gmtime, localtime, etc would be way too big to fit on the mcu. I was just thinking of parsing the NMEA time and setting the bcd digits of the time in the rtc I agree it may be difficult to fit the code for for any advanced time handling. So through make defines should be ok for now like you proposed. (I guess later if we can figure out how to do it, we can expose hour offset through ui.) There are some tricks we can play by putting big functions/tables/vars into "eeprom" space, as it should still be addressible/callable via extended access, from my recollection with experimenting on stc15w404as and stc15f204ea

Also as i think we are using sdcc memory-model small, it tries to put everything in data/idata, but it has additional 256b xdata ram space as well, you just have to decorate your various large string buffers/etc with this. (again, we start getting into the area where you have to play tricks to free up data space for the stack, and more advanced: placing tables/strings/ (maybe) functions into "eeprom" code space)

So if you have something functional now, I can take that, and we can optimize/refine/enhance later on. Thanks again for the hard work on this.

[zerog2k]

also curious, did you use sdcc's time.h? https://sourceforge.net/p/sdcc/code/HEAD/tree/trunk/sdcc/device/include/time.h