Provide 64-bit time_t support on 32-bit archs

[adrien-n]

Hi,

When looking at issues in datefudge (same purpose as faketime but more limited scope) with the update to coreutils >= 9 in Ubuntu, it appeared that there is no support for binaries using using 64-bit time_t on machines with a 32-bit time_t by "default".

Such binaries use symbols suffixed with "64" as in "__clock_gettime64@GLIBC_2.34".

NB: You can get an i386 build of coreutils that exhibit this as http://launchpadlibrarian.net/645483093/coreutils_9.1-1ubuntu2_i386.deb .

It is likely that applications are going to more and more switch to 64-bit time_t even on 32-bit archs. This matters especially for ones that handle TLS certificates and which are also already users of faketime (or datefudge).

Considering this is likely going to require additional code, it is likely that this will be done in faketime and that users of datefudge will be redirected to faketime (basically gnutls and oath-toolkit).

We're looking to have a fix ready within 8 months at most because faketime is used for a lot of packages.

Was this issue already thought about? Do faketime developers wish to support this? Does anyone have an idea on how to best support such binaries?

So far we haven't really experimented with patches for this in Ubunt but one possibly approach we imagined is basically to build libfaketime.c twice, one for each ABI and ship both in a single library. Thre will be more things to do obviously but from an outside point-of-view, this sounds like a possible way to avoid duplicating too much code. By the way, a new scenario to keep in mind will be binaries using both ABIs and binaries using one ABI and spawning binaries that use the other. There a number of new scenarios that could arise too but that can't make sense(for instance, going over y2038 and exchanging values across ABIs).

Thanks.

[wolfcw]

Thanks for pointing this out. The necessity to support those 64-bit time_t function variants on 32-bit machines is crystal clear, yet so far there was a lack of awareness for the problem on this end.

Avoiding lots of code duplication would be smart, but not necessarily top priority. Eventually similarities to how xstat() / xstat64() and friends are already handled could be leveraged.

[adrien-n]

We uncovered this issue very recently: updating coreutils automatically triggered tests in datefudge, including on 32bit archs.

I'll have a look at xstat*. Thanks.

I also plan to add crude auditing to faketime in order to be able to learn which functions we need first (I don't think there's something like that at the moment).

[adrien-n]

I finally had an actual try at this.

An important question: which libc should be supported? I'm not sure it makes sense to support any but glibc at the moment.

And when supporting glibc, the issues I've hit is that its implementation has changed and that it now implements the interface for 32-bit time_t as a wrapper around the 64-bit one which is always implemented (it's possible to not have the 32-bit one but it's not possible to not have the 64-bit one). Unfortunately, it's not possible to intercept the calls inside glibc since they are resolved statically inside the library. This means we need to provide the conversion functions.

The conversion functions are not difficult to write but glibc doesn't give access to the types it uses. I've resorted to building libfaketime.c with 64-bit time_t and implement the conversion code in another .c file that uses 64-bit time_t. It's not very pretty, especially at the boundary between the two .c files but I think it will reduce errors because it makes it more difficult to mix types unknowingly.

So far I've done the work for clock_gettime and I could fake epoch 0. I hope the groundwork will be appropriate for other functions too.

I have a couple questions by the way.

If FAKE_INTERNAL_CALLS, why have both clock_gettime() and __clock_gettime()?

Are you okay with copying code from glibc. I prefer to ask even though the licenses match and most of the corresponding glibc code probably doesn't pass copyrightability conditions because it's trivial and impossible to write differently, and the rest is type definitions which are absolutely required for compatibility.

[wolfcw]

Thanks for diving into this!

As far as libc compatibility is concernced, the usual path would be:

• Start with supporting the current version of glibc.
• If the changes break compatibility with older glibc versions, please try to wrap them in #ifdef's for a new compile-time flag, which we can enable by default.
• Other libc implementations are of secondary concern, but if you expect trouble there, it again would be nice to wrap the changes as above so they can be disabled at compile-time.

Also, I consider it OK to copy code from glibc as you outlined above. libfaketime should stay compilable without having glibc sources on the building system. I suggest you proceed and we then check whether the result looks critical under that aspect, but I wouldn't expect that.

The FAKE_INTERNAL_CALLS compile-time flag and the few functions it affects was once (as in like 10 years ago) required to make libfaketime work with Sun's / Oracle's Java Virtual Machine / runtime, which apparently deliberately called the __variants of the standard libc functions. It's still enabled by default as it typically doesn't hurt (hardly any application wil call it, and if it does, faking the time is the way to go), but there's no telemetry on whether it's still in use/required for more recent JREs.

[adrien-n]

I haven't touched that yet but I'm a bit concerned about compatibility between glibc versions. I don't know how this will pan out but I guess we'll only know after working on it.

And thanks for the background on the variants. I'm not sure clock_gettime64() is "compatible" with what the JVM is/was using and there might be incompatibilities there too but I can't easily find references to it in the jdk sources and they might have stopped using it.

I don't know at which rate I'll be able to work on this; I'll prepare a PR soon-ish; it definitely won't be mergeable at that point though.

[adrien-n]

Ubuntu (and maybe Debian) is likely to migrate in the rather short-term to a 64-bit time_t on armhf (i386 will continue not to evolve). Considering this change should happen rather sooner than later, having support for both time_t sizes at once in faketime doesn't seem required (it's possible to deal with or ignore the errors for a few months considering it's mostly coreutils which has moved to 64-bit time_t at the moment). Overall the time and difficulty to implement what I've described above probably outweigh the advantages.

I'd like to clean up a bit my current work and share it if others are interested in continuing the work. I don't know if many besides a few distributions have workloads where the size of time_t isn't known or is mixed (as can happen after exec()).

I see value in being able to handle both sizes on the same system however and I think this can be more efficiently and more safely handled by having two faketime installations or two libfaketime installations (I think it's not possible to install two versions of the library at the moment).