zero-valued timestamps are mishandled by os.stat() in Windows

[b17c4d67-f663-4928-bfd4-8bc1635306fd]

BPO	23946
Nosy	@pfmoore, @tjguk, @zware, @eryksun, @zooba, @CristiFati, @iritkatriel
Files	Python-2.7.8-invalid_filetimes.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', '3.8', '3.9', '3.10', 'library', 'OS-windows'] title = 'zero-valued timestamps are mishandled by os.stat() in Windows' updated_at = user = 'https://github.com/CristiFati' ``` bugs.python.org fields: ```python activity = actor = 'eryksun' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)', 'Windows'] creation = creator = 'CristiFati' dependencies = [] files = ['38985'] hgrepos = [] issue_num = 23946 keywords = ['patch'] message_count = 4.0 messages = ['240875', '382223', '382251', '388116'] nosy_count = 7.0 nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'CristiFati', 'iritkatriel'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue23946' versions = ['Python 3.8', 'Python 3.9', 'Python 3.10'] ```

[b17c4d67-f663-4928-bfd4-8bc1635306fd]

In WinPE environment (at least this is where I found it, but there might be cases where this issue could be spotted in normal Windows), some folders might have some of the FILETIME structures reset (because of RAMDrive?). When the conversion between WIN style (100 * nsecs from 1601/01/01) to UX style (secs from 1970/01/01) is taking place (epoch is being substracted), it could yield invalid filetime values (that will be reported by os.stat).

The attached patch simply goes around this initializing the UX timestamps to 0.

It was found in Python-2.7.3, the patch is for 2.7.8, but I think it can be reproduced with all the newer versions.

[iritkatriel]

The code of posixmodule.c looks very different now. It this issue still relevant?

[eryksun]

The code of posixmodule.c looks very different now.

The conversion code was moved to FILE_TIME_to_time_t_nsec() in Python/fileutils.c.

time_t is a signed 64-bit integer type, so there's no immediate problem storing 1601-01-01 as the negative (pre-epoch) Unix timestamp -11644473600. But there are indirect problems with using negative Unix timestamps in Windows. In particular, datetime.fromtimestamp doesn't support them:

    >>> datetime.fromtimestamp(-11644473600)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 22] Invalid argument

[eryksun]

For example, the named-pipe filesystem (NPFS), doesn't return timestamps for pipes in the directory listing, so the timestamps are all 0 (i.e. 1601-01-01):

    >>> write_time = win32file.FindFilesW('//./pipe/*')[0][3]
    >>> format(write_time, '%Y-%m-%d %H:%M')
    '1601-01-01 00:00'

    >>> next(os.scandir('//./pipe')).stat().st_mtime
    -11644473600.0

I agree that a zero-valued NT timestamp should be converted to a zero-valued Unix timestamp. No one has a file that was created, modified, changed, or accessed in 1601.

As mentioned above, the 1601 date doesn't roundtrip in Windows as a Unix timestamp since dates before the Unix epoch aren't supported:

    >>> write_time.timestamp()
    -11644473600.0
    >>> datetime.fromtimestamp(write_time.timestamp())
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 22] Invalid argument