Libc no longer handles daylight savings time.

[dryeo]

The Mozilla apps have stopped recognizing daylight saving time (DST), in my case, emails showing the offset as -0800 instead of the correct -0700. This worked last year and is broken no matter how we back level stuff until Mozilla 1.4 built with VACCP. This is discussed in https://mantis.arcanoae.com/view.php?id=3158 including a testcase. Setting the date to a year ago, I get

H:\tmp>TestDosGetDateTimeTzSetAndRuntime_gcc.exe

testdosgetdatetimetzsetandruntime_gcc built on Jun 12 2013 14:37:44 with GCC 4.4.7

DosGetDateTime says the time is 21:08:41:01 and the date is 2020/7/17
DosGetDateTime says the GMT offset is is -1 (i.e. unset) and the weekday is Fri(5)

The TZ variable is set to "PST8PDT,3,2,0,7200,11,1,0,7200,3600"
Issuing tzset()

The runtime (tzset) has set the tzname array to PST PDT
The runtime has set the timezone variable 8:00:00 hours
The runtime has set the daylight flag to 1
The runtime says local time is Fri Jul 17 21:08:41 2020
The runtime says local time is observing DST

The runtime says GMT is Sat Jul 18 04:08:41 2020
The runtime says GMT is observing DST

The GMT offset calculated using mktime results is -0700
The GMT offset calculated using runtime timezone value and local isdst value -0700

While now it is,

H:\tmp>TestDosGetDateTimeTzSetAndRuntime_gcc.exe

testdosgetdatetimetzsetandruntime_gcc built on Jun 12 2013 14:37:44 with GCC 4.4.7

DosGetDateTime says the time is 21:22:31:00 and the date is 2021/7/17
DosGetDateTime says the GMT offset is is -1 (i.e. unset) and the weekday is Sat(6)

The TZ variable is set to "PST8PDT,3,2,0,7200,11,1,0,7200,3600"
Issuing tzset()

The runtime (tzset) has set the tzname array to PST PDT
The runtime has set the timezone variable 8:00:00 hours
The runtime has set the daylight flag to 1
The runtime says local time is Sat Jul 17 21:22:31 2021
The runtime says local time is not observing DST

The runtime says GMT is Sun Jul 18 05:22:31 2021
The runtime says GMT is not observing DST

The GMT offset calculated using mktime results is -0800
The GMT offset calculated using runtime timezone value and local isdst value -0800

Other apps such as the VirtualBox additions are also affected.

[SilvanScherrer]

Please attach the source of the testcase. So we can build it ourself and test

[dmik]

What makes you think "no longer"? Do yo have some older LIBC DLL that works in your case? As a wild guess, this may be related to outdated TZ data somehow (and the fact that some library started using it instead of some older method). They change TZ settings pretty much often nowadays around the world.

[StevenLevine]

Here you go. I rebuilt TestDosGetDateTimeTzSetAndRuntime_gcc.exe with the latest bww gcc, but this did not seem to change the results.

[StevenLevine]

I cannot tell if github deigned to upload the zip file. It can be found at http://www.warpcave.com/betas/TestDosGetDateTimeTzSetAndRuntime-20210718.zip

[dryeo]

By no longer, I mean this year all libc Mozilla apps have broken. This includes SM 2.5 (FF7 based) with a real libc063 and SM 1.17 (FF2.x), which is pre-XUL.DLL. Mozilla 1.4 correctly seems to handle DST. For the older builds, the test is evaluating the JS Date() function, which is also currently broken. The only libraries they have in common is various libc builds, with the older ones statically linking the GCC DLLs. The latest seem to use libicu for handling timezones, perhaps just localization of them.

[dmik]

@dryeo do you mean that you see the problem when running the test case against LIBC063 as well?

On my system (Moscow time UTC+3, no DST since 2014) the output of VAC, WAT, GCC is, respectively:

testdosgetdatetimetzsetandruntime_vac built on Jun 12 2013 14:36:38 with VAC 3.65
DosGetDateTime says the time is 17:27:50:84 and the date is 2021/7/19
DosGetDateTime says the GMT offset is is -1 (i.e. unset) and the weekday is Mon (1)

The TZ variable is set to "MSK-3MSD,3,-1,0,7200,10,-1,0,7200,3600"
Issuing tzset()

The runtime (tzset) has set the tzname array to MSK MSD
The runtime has set the timezone variable -3:00:00 hours
The runtime has set the daylight flag to 1
The runtime says local time is Mon Jul 19 17:27:50 2021
The runtime says local time is observing DST

The runtime says GMT is Mon Jul 19 14:27:50 2021
The runtime says GMT is observing DST

The GMT offset calculated using mktime results is +0300
The GMT offset calculated using runtime timezone value and local isdst value +0400

testdosgetdatetimetzsetandruntime_wat built on Jun 12 2013 14:30:44 with __WATCOMC__ 2.0 (2000)

DosGetDateTime says the time is 17:27:55:94 and the date is 2021/7/19
DosGetDateTime says the GMT offset is is -1 (i.e. unset) and the weekday is Mon (1)

The TZ variable is set to "MSK-3MSD,3,-1,0,7200,10,-1,0,7200,3600"
Issuing tzset()

The runtime (tzset) has set the tzname array to MSK MSD
The runtime has set the timezone variable -3:00:00 hours
The runtime has set the daylight flag to 1
The runtime says local time is Mon Jul 19 17:27:56 2021
The runtime says local time is observing DST

The runtime says GMT is Mon Jul 19 13:27:56 2021
The runtime says GMT is not observing DST

The GMT offset calculated using mktime results is +0300
The GMT offset calculated using runtime timezone value and local isdst value +0400

testdosgetdatetimetzsetandruntime_gcc built on Jul 18 2021 09:05:37 with GCC 9.2.0 20190812 (OS/2 RPM build 9.2.0-5.oc00)

DosGetDateTime says the time is 17:28:00:07 and the date is 2021/7/19
DosGetDateTime says the GMT offset is is -1 (i.e. unset) and the weekday is Mon (1)

The TZ variable is set to "MSK-3MSD,3,-1,0,7200,10,-1,0,7200,3600"
Issuing tzset()

The runtime (tzset) has set the tzname array to MSK MSD
The runtime has set the timezone variable -3:00:00 hours
The runtime has set the daylight flag to 1
The runtime says local time is Mon Jul 19 17:28:00 2021
The runtime says local time is not observing DST

The runtime says GMT is Mon Jul 19 14:28:00 2021
The runtime says GMT is not observing DST

The GMT offset calculated using mktime results is +0300
The GMT offset calculated using runtime timezone value and local isdst value +0300

The mktime value is the correct one in all cases, the "runtime" thing is only correct for the GCC case and wrong for VAC and WAT. I will look closer at the sources. But it's already clearly seen that GCC (kLIBC/LIBCn) ignores the non-standard TZ format extension specifying the DST start/stop. Under GCC I see that the daylight flag is reported as "set" but local and GMT times don't take it into account.

Both VAC and GCC however, correctly report GMT (i.e. VAC ignores DST while still reports it in the tm struct whcih GCC does not do). Kind of weird.

[dryeo]

I only did the evaluating JS Date() with libc063. Here's the testcase using libc065

testdosgetdatetimetzsetandruntime built on Jul 19 2021 07:02:04 with GCC 3.3.5 (
Bird Build 2012-03-23 03:21 (csd5))

DosGetDateTime says the time is 07:02:37:26 and the date is 2021/7/19
DosGetDateTime says the GMT offset is is -1 (i.e. unset) and the weekday is Mon(1)

The TZ variable is set to "PST8PDT,3,2,0,7200,11,1,0,7200,3600"
Issuing tzset()

The runtime (tzset) has set the tzname array to PST PDT
The runtime has set the timezone variable 8:00:00 hours
The runtime has set the daylight flag to 1
The runtime says local time is Mon Jul 19 07:02:37 2021
The runtime says local time is not observing DST

The runtime says GMT is Mon Jul 19 15:02:37 2021
The runtime says GMT is not observing DST

The GMT offset calculated using mktime results is -0800
The GMT offset calculated using runtime timezone value and local isdst value -0800

I also tried the testcase with EMX (GCC 2.81) where,

The GMT offset calculated using mktime results is -0700
The GMT offset calculated using runtime timezone value and local isdst value -0600 

[dmik]

Okay, I know what's wrong. Despite what I said above, kLIBC does the right thing when calculating DST from the VAC-compatible TZ format (which isn't Posix compatible and handling Posix compatible TZ is simply missing in kLIBC yet (see https://github.com/bitwiseworks/libc/blob/efb19af664b670a6ef78f702eb3acd1a670c70e5/src/emx/src/lib/time/tzset.c#L257).

It a classical year 2038 bug in kLIBC which is there since years (forever?). For years 2038 and above, the offset in seconds since 01.01.1970 overflows a 32 bit integer (time_t). This would break filling up the dst table in _compute_dst_table (this table contains moments in seconds since 01.01.1970 where DST starts and ends). Starting from year 2038 this table would get polluted with wrong negative values so a later search for a right moment using binary search would return an end of list marker which is treated as no DST.

Note that before year 2021 the binary search would just never get into that wrong part of the dst table so all worked as expected.

Also note that there is a similar problem with the 32 bit integer underflow caused by years 1900 and 1901 but it was specifically accounted for in _compute_dst_table (by simply excluding these years so that they would never get reported as having DST regardless of TZ). But apparently the author who did it in 1990's simply didn't care about year 2038 and above hence the bug.

I will try to fix it nicely (including years 1900 and 1901) by using a 64-bit time_t variant (time64_t, which is already in kLIBC).

Note, however, that time.t defines a _YEARS constant which assumes that kLIBC covers years since 1900 to 2059 (inclusive) when doing all this DST math in localtime, gmtime and friends. This means that at some point we will face year 2059 problem where DST will stop working again... A simple solution is to increase _YEARS to include more years. This should even be backward compatible with older software (as internal arrays will grow, not shrink). I will create a separate ticket for that after resolving this one.

[dmik]

With the above commit all works as it should:

testdosgetdatetimetzsetandruntime_gcc built on Jul 18 2021 09:05:37 with GCC 9.2.0 20190812 (OS/2 RPM build 9.2.0-5.oc00)

DosGetDateTime says the time is 23:38:42:53 and the date is 2021/7/19
DosGetDateTime says the GMT offset is is -1 (i.e. unset) and the weekday is Mon (1)

The TZ variable is set to "MSK-3MSD,3,-1,0,7200,10,-1,0,7200,3600"
Issuing tzset()

The runtime (tzset) has set the tzname array to MSK MSD
The runtime has set the timezone variable -3:00:00 hours
The runtime has set the daylight flag to 1
The runtime says local time is Mon Jul 19 23:38:42 2021
The runtime says local time is observing DST

The runtime says GMT is Mon Jul 19 19:38:42 2021
The runtime says GMT is observing DST

The GMT offset calculated using mktime results is +0400
The GMT offset calculated using runtime timezone value and local isdst value +0400

As one may see both mkime and and local isdst are consistent, as opposed to VAC and WAT (some bugs there? might be some sort of Y2038 bug as well).

A test drop can be found as usual at http://rpm.netlabs.org/test/libcn0.7z. Please test.

PS. Of course now I had to replace my TZ in CONFIG.SYS with just TZ=MSK-3 to be in the right TZ (no DST in Russia since 2014).

[altsan]

After running several tests with this build, it seems to fix the issue in Virtualbox Additions.

With TZ=EST5EDT,3,2,0,7200,11,1,0,7200,3600 --> Time is now correct (was previously 1hr behind) With TZ=EST5EDT --> Time is now correct (was previously 1hr behind) With TZ=EST5 --> Time is 1hr behind (DST disabled)

Looking good here.

[StevenLevine]

The update seems to fix most of the oddities here in PST8PDT.

The VAC runtime is a bit confused because it thinks GMT is observing DST, but given it's age this is not unexpected.

My question is why does the kLIBC runtime think GMT is observing DST?

[dryeo]

Verified as fixed here. Thank you. I see the JavaScript Date() still thinks it is PST, now with the right offset. This must be a bug in the Mozilla code as it appeared between SM 1.18 (working correctly) and SM 2.5 (Gecko 8) where it is broken.

[dmik]

Thanks everyone for testing.

My question is why does the kLIBC runtime think GMT is observing DST?

@StevenLevine it's not actually true. After calling gmtime your code also calls mktime on the same struct. gmtime in kLIBC does not fiddle with tm_isdst and just sets it to -1 ("info not available", per POSIX — which is rather correct to me). But for mktime -1 means determine DST from the environment — and mktime does that fairly from the TZ setting: the specified moment in time is under DST according to it. And hence it sets tm_isdst to 1 on output. Note that the way you do it in your test is an "abuse" of mktime: strictly speaking, it expects to receive local time on input, not GMT. If you are sure you feed it with GMT, you should deliberately set tm_isdst to 0. This is how I read POSIX specs.

Also note that you should check for tm_isdst == 1 when checking for DST, not just tm_isdst, because it might be legitimately set to -1 ("info not available") which such a code would wrongly interpret as DST in effect.

So, to me, kLIBC/LIBCn is fully correct in this regard. While VAC and WAT don't fully match POSIX.

[dmik]

I created a separate ticket for Y2060 problem and closing this one.

[dryeo]

Be nice for a RPM to be built containing this fix and ideally fast tracked to netlabs-rel