ts.utc_strftime() milliseconds not working on Python 3.9 on Windows

[aendie]

The following is sample test code in Python 2 with Skyfield 1.30:

import datetime
from skyfield.api import load, utc

ts = load.timescale()   # timescale object
PreviousNewMoon = datetime.datetime(2022, 11, 23, 22, 57, 13, 903886, tzinfo=utc)

t2 = ts.utc(PreviousNewMoon)
print("t2 = " + t2.utc_strftime(format='%Y-%m-%d %H:%M:%S UTC'))
print("t2 = " + t2.utc_strftime(format="%f"))

The first print line is identical to the documentation and works. However attempting to print microseconds fails:

The Python 2 datetime docs: https://docs.python.org/2.7/library/datetime.html include the following format definition for %f:

---

The following is sample test code in Python 3 with Skyfield 1.37:

import datetime
from skyfield.api import load, utc

ts = load.timescale(builtin=False)  # timescale object
PreviousNewMoon = datetime.datetime(2022, 11, 23, 22, 57, 13, 903886, tzinfo=utc)

t2 = ts.utc(PreviousNewMoon)
print("t2 = " + t2.utc_strftime(format='%Y-%m-%d %H:%M:%S UTC'))
print("t2 microsec = " + t2.utc_strftime(format="%f"))

The first print line is identical to the documentation and works. However attempting to print microseconds prints "W. Europe Standard Time" instead:

The Python 3 datetime docs: https://docs.python.org/3/library/datetime.html include the following format definition for %f:

---

This looks like a minor bug to me, however ... praise where praise is due:

I would like to add how much I appreciate the excellent Skyfield documentation in general. In particular I like this page: https://rhodesmill.org/skyfield/searches.html and the style - including the "Well, drat." and the "Much better!" comments. That's just the way documentation should be written!

[brandon-rhodes]

sample test code in Python 2

I have just landed a commit to improve the exception raised in Python 2 so that it says: strftime() "%f" not supported under Python 2

sample test code in Python 3

Hmm, that's a more serious problem, since your example code works fine on my laptop. What version of Python 3 are you using?

I would like to add how much I appreciate the excellent Skyfield documentation in general.

Thank you! It's heartening to know you've often found it useful, and that I have not been amiss in adding a bit of drama to the language when illustrating problem solving.

[aendie]

Python 3.9.1 I can also try the same test on my laptop tomorrow.

[aendie]

Well now ... I have a dual boot PC with Ubuntu 20.04 and it works correctly there:

I have a laptop with a new system drive - a Seagate SSD that has a clean Windows 10 Pro installation (and no Python yet). I installed Python 3.9.1 - and it failed printing "W. Europe Standard Time". Then I uninstalled it and I installed Python 3.9.2 - and it failed printing "W. Europe Standard Time".

So I experimented a bit with the python trace function on my PC (in Windows 10). Attached here is an excerpt from the end. On line 96 below the print instruction is traced. I am unfamiliar with traces, so it says nothing to me. I am wondering if this should be reported as a Python bug (which I've never done before)?

issue562.py(12): PreviousNewMoon = datetime.datetime(2022, 11, 23, 22, 57, 13, 903886, tzinfo=utc)
issue562.py(14): t2 = ts.utc(PreviousNewMoon)
 --- modulename: timelib, funcname: utc
timelib.py(141):         if isinstance(year, datetime):
timelib.py(142):             return self.from_datetime(year)
 --- modulename: timelib, funcname: from_datetime
timelib.py(124):         return self._utc(_datetime_to_utc_tuple(datetime))
 --- modulename: timelib, funcname: _datetime_to_utc_tuple
timelib.py(1007):     z = dt.tzinfo
timelib.py(1008):     if z is None:
timelib.py(1010):     if z is not utc:
timelib.py(1012):     return (dt.year, dt.month, dt.day,
timelib.py(1013):             dt.hour, dt.minute, dt.second + dt.microsecond / 1e6)
timelib.py(1012):     return (dt.year, dt.month, dt.day,
 --- modulename: timelib, funcname: _utc
timelib.py(152):         year, month, day, hour, minute, second = tup
timelib.py(153):         whole, fraction = self._jd(year, month, day, hour, minute, 0.0)
 --- modulename: timelib, funcname: _jd
timelib.py(162):         a = _to_array
timelib.py(163):         cutoff = self.julian_calendar_cutoff
timelib.py(164):         whole = julian_day(a(year), a(month), a(day), cutoff) - 0.5
 --- modulename: functions, funcname: _to_array
functions.py(156):     if hasattr(value, 'shape'):
functions.py(158):     elif hasattr(value, '__len__'):
functions.py(161):         return float64(value)
 --- modulename: functions, funcname: _to_array
functions.py(156):     if hasattr(value, 'shape'):
functions.py(158):     elif hasattr(value, '__len__'):
functions.py(161):         return float64(value)
 --- modulename: functions, funcname: _to_array
functions.py(156):     if hasattr(value, 'shape'):
functions.py(158):     elif hasattr(value, '__len__'):
functions.py(161):         return float64(value)
 --- modulename: timelib, funcname: julian_day
timelib.py(848):     y, month = divmod(month - 1, 12)
timelib.py(849):     year = year + y
timelib.py(850):     month += 1
timelib.py(853):     janfeb = month <= 2
timelib.py(854):     g = year + 4716 - janfeb
timelib.py(855):     f = (month + 9) % 12
timelib.py(856):     e = 1461 * g // 4 + day - 1402
timelib.py(857):     J = e + (153 * f + 2) // 5
timelib.py(859):     mask = 1 if (julian_before is None) else (J >= julian_before)
timelib.py(860):     J += (38 - (g + 184) // 100 * 3 // 4) * mask
timelib.py(861):     return J
timelib.py(165):         fraction = (a(second) + a(minute) * 60.0 + a(hour) * 3600.0) / DAY_S
 --- modulename: functions, funcname: _to_array
functions.py(156):     if hasattr(value, 'shape'):
functions.py(158):     elif hasattr(value, '__len__'):
functions.py(161):         return float64(value)
 --- modulename: functions, funcname: _to_array
functions.py(156):     if hasattr(value, 'shape'):
functions.py(158):     elif hasattr(value, '__len__'):
functions.py(161):         return float64(value)
 --- modulename: functions, funcname: _to_array
functions.py(156):     if hasattr(value, 'shape'):
functions.py(158):     elif hasattr(value, '__len__'):
functions.py(161):         return float64(value)
timelib.py(166):         return _reconcile(whole, fraction)
 --- modulename: functions, funcname: _reconcile
functions.py(165):     an = getattr(a, 'ndim', 0)
functions.py(166):     bn = getattr(b, 'ndim', 0)
functions.py(167):     difference = bn - an
functions.py(168):     if difference > 0:
functions.py(173):     elif difference < 0:
functions.py(178):     return a, b
timelib.py(154):         i = searchsorted(self.leap_dates, whole + fraction, 'right')
 --- modulename: fromnumeric, funcname: _searchsorted_dispatcher
fromnumeric.py(1278):     return (a, v, sorter)
 --- modulename: fromnumeric, funcname: searchsorted
fromnumeric.py(1348):     return _wrapfunc(a, 'searchsorted', v, side=side, sorter=sorter)
 --- modulename: fromnumeric, funcname: _wrapfunc
fromnumeric.py(53):     bound = getattr(obj, method, None)
fromnumeric.py(54):     if bound is None:
fromnumeric.py(57):     try:
fromnumeric.py(58):         return bound(*args, **kwds)
timelib.py(155):         fraction += (self.leap_offsets[i] + second) / DAY_S
timelib.py(156):         whole, fraction = _reconcile(whole, fraction)  # second could be array
 --- modulename: functions, funcname: _reconcile
functions.py(165):     an = getattr(a, 'ndim', 0)
functions.py(166):     bn = getattr(b, 'ndim', 0)
functions.py(167):     difference = bn - an
functions.py(168):     if difference > 0:
functions.py(173):     elif difference < 0:
functions.py(178):     return a, b
timelib.py(157):         t = Time(self, whole, fraction + tt_minus_tai)
 --- modulename: timelib, funcname: __init__
timelib.py(330):         if tt_fraction is None:
timelib.py(332):         self.ts = ts
timelib.py(333):         self.whole = tt
timelib.py(334):         self.tt_fraction = tt_fraction
timelib.py(335):         self.shape = getattr(tt, 'shape', ())
timelib.py(158):         t.tai_fraction = fraction
timelib.py(159):         return t
issue562.py(16): print("t2 microsec = " + t2.utc_strftime(format="%f"))
 --- modulename: timelib, funcname: utc_strftime
timelib.py(542):         offset, uses_ms = _strftime_offset_seconds(format)
 --- modulename: timelib, funcname: _strftime_offset_seconds
timelib.py(1024):     uses_ms = _format_uses_milliseconds(format)
timelib.py(1025):     if uses_ms:
timelib.py(1026):         offset = 1e-16  # encourage .0 to not turn into .999999
timelib.py(1033):     return offset, uses_ms
timelib.py(543):         year, month, day, hour, minute, second, jd = self._utc_tuple(offset, 1)
 --- modulename: timelib, funcname: _utc_tuple
timelib.py(562):         second, sfr, is_leap_second = self._utc_seconds(offset)
 --- modulename: timelib, funcname: _utc_seconds
timelib.py(586):         seconds, fr = self._tai_seconds
 --- modulename: descriptorlib, funcname: __get__
descriptorlib.py(10):         if instance is None:
descriptorlib.py(12):         value = self.method(instance)
 --- modulename: timelib, funcname: _tai_seconds
timelib.py(598):         seconds, fr = divmod(self.whole * DAY_S, 1.0)
timelib.py(599):         seconds2, fr = divmod(fr + self.tai_fraction * DAY_S, 1.0)
timelib.py(600):         seconds += seconds2
timelib.py(601):         return seconds, fr
descriptorlib.py(13):         instance.__dict__[self.__name__] = value
descriptorlib.py(14):         return value
timelib.py(587):         seconds2, fr = divmod(fr + offset, 1.0)
timelib.py(588):         seconds = seconds + seconds2  # not +=, which would modify cached array
timelib.py(589):         ts = self.ts
timelib.py(590):         tai_minus_utc = interp(seconds, ts._leap_tai, ts._leap_offsets)
 --- modulename: function_base, funcname: _interp_dispatcher
function_base.py(1287):     return (x, xp, fp)
 --- modulename: function_base, funcname: interp
function_base.py(1395):     fp = np.asarray(fp)
 --- modulename: _asarray, funcname: asarray
_asarray.py(99):     if like is not None:
_asarray.py(102):     return array(a, dtype, copy=False, order=order)
function_base.py(1397):     if np.iscomplexobj(fp):
 --- modulename: type_check, funcname: _is_type_dispatcher
type_check.py(207):     return (x,)
 --- modulename: type_check, funcname: iscomplexobj
type_check.py(312):     try:
type_check.py(313):         dtype = x.dtype
type_check.py(314):         type_ = dtype.type
type_check.py(317):     return issubclass(type_, _nx.complexfloating)
function_base.py(1401):         interp_func = compiled_interp
function_base.py(1402):         input_dtype = np.float64
function_base.py(1404):     if period is not None:
function_base.py(1428):     return interp_func(x, xp, fp, left, right)
timelib.py(591):         tai_minus_utc, is_leap_second = divmod(tai_minus_utc, 1.0)
timelib.py(592):         is_leap_second = is_leap_second > 0.0
timelib.py(593):         return seconds - tai_minus_utc, fr, is_leap_second
timelib.py(563):         second = second.astype(int64)
timelib.py(564):         second -= is_leap_second
timelib.py(565):         jd, second = divmod(second + 43200, 86400)
timelib.py(566):         cutoff = self.ts.julian_calendar_cutoff
timelib.py(567):         year, month, day = compute_calendar_date(jd, cutoff)
 --- modulename: timelib, funcname: compute_calendar_date
timelib.py(885):     use_gregorian = (julian_before is None) or (jd_integer >= julian_before)
timelib.py(888):     f = jd_integer + 1401
timelib.py(889):     f += use_gregorian * ((4 * jd_integer + 274277) // 146097 * 3 // 4 - 38)
timelib.py(890):     e = 4 * f + 3
timelib.py(891):     g = e % 1461 // 4
timelib.py(892):     h = 5 * g + 2
timelib.py(893):     day = h % 153 // 5 + 1
timelib.py(894):     month = (h // 153 + 2) % 12 + 1
timelib.py(895):     year = e // 1461 - 4716 + (12 + 2 - month) // 12
timelib.py(896):     return year, month, day
timelib.py(568):         minute, second = divmod(second, 60)
timelib.py(569):         hour, minute = divmod(minute, 60)
timelib.py(570):         second += is_leap_second
timelib.py(571):         if not return_jd:
timelib.py(573):         return year, month, day, hour, minute, second + sfr, jd
timelib.py(544):         start_of_year = julian_day(year, 1, 1, self.ts.julian_calendar_cutoff)
 --- modulename: timelib, funcname: julian_day
timelib.py(848):     y, month = divmod(month - 1, 12)
timelib.py(849):     year = year + y
timelib.py(850):     month += 1
timelib.py(853):     janfeb = month <= 2
timelib.py(854):     g = year + 4716 - janfeb
timelib.py(855):     f = (month + 9) % 12
timelib.py(856):     e = 1461 * g // 4 + day - 1402
timelib.py(857):     J = e + (153 * f + 2) // 5
timelib.py(859):     mask = 1 if (julian_before is None) else (J >= julian_before)
timelib.py(860):     J += (38 - (g + 184) // 100 * 3 // 4) * mask
timelib.py(861):     return J
timelib.py(545):         weekday = jd % 7
timelib.py(546):         yday = jd + 1 - start_of_year
timelib.py(547):         return _strftime(format, year, month, day, hour, minute, second,
timelib.py(548):                          weekday, yday, uses_ms)
timelib.py(547):         return _strftime(format, year, month, day, hour, minute, second,
 --- modulename: timelib, funcname: _strftime
timelib.py(1037):     zero = year * 0
timelib.py(1042):     if uses_ms:
timelib.py(1043):         format = format[:uses_ms.start()] + '%Z' + format[uses_ms.end():]
timelib.py(1044):         second = (second * 1e6).astype(int)
timelib.py(1045):         second, usec = divmod(second, 1000000)
timelib.py(1046):         if getattr(year, 'ndim', 0):
timelib.py(1051):         u = '%06d' % usec
timelib.py(1052):         tup = year, month, day, hour, minute, second, weekday, yday, zero, u
timelib.py(1053):         return strftime(format, struct_time(tup))
t2 microsec = W. Europe Standard Time
issue562.py(17): sys.exit(0)

D:\_DEVelopment\Astronomical coding\_Issue 562\Py3>

Hah! A few lines from the end I see timelib.py(1043): format = format[:uses_ms.start()] + '%Z' + format[uses_ms.end():] and %Z is

I don't have time just now to attempt a trace in Ubuntu.

[brandon-rhodes]

Because Python's time.strftime() doesn't support microseconds, Skyfield replaces %f with %Z in the format string and then provides a time zone name like 903886 or whatever the microseconds are. I have just added Python 3.9 to Skyfield's test matrix, fearing that maybe that broke under Python 3.9, but the tests are passing fine.

A print(tup) added to Skyfield’s code right above return strftime(…) would confirm whether the tuple itself at least is being built correctly. And would let you create a 3-line test repro:

from time import strftime, struct_time
tup = (... the value printed out ...)
print(strftime('%Z', struct_time(tup)))

For combinations of OS and Python where it prints the milliseconds field, all is well. You might have found a new combination, though, where %Z ignores the timezone passed in the tuple?

[brandon-rhodes]

@aendie — Following up on this issue again: did you find that this problem continued, with more recent versions of Python under Windows, or did the problem go away on its own?

[aendie]

TEST PERFORMED ON English Windows 10 PC WITH Python 3.10

I use this (current) version of Windows 10:

And with Python version 3.10.8 the problem still exists:

TEST REPEATED ON English Windows 10 LAPTOP WITH Python 3.11

Here again it fails:

TEST REPEATED ON German Windows 11 LAPTOP WITH Python 3.11

Here again it fails: