largefile support for Win64 (and some other fixes)

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

BPO	400510
Nosy	@gvanrossum, @tim-one, @freddrake
Files	None: None

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['interpreter-core'] title = 'largefile support for Win64 (and some other fixes)' updated_at = user = 'https://bugs.python.org/tmick' ``` bugs.python.org fields: ```python activity = actor = 'tmick' assignee = 'tmick' closed = True closed_date = None closer = None components = ['Interpreter Core'] creation = creator = 'tmick' dependencies = [] files = ['2441'] hgrepos = [] issue_num = 400510 keywords = ['patch'] message_count = 9.0 messages = ['32708', '32709', '32710', '32711', '32712', '32713', '32714', '32715', '32716'] nosy_count = 4.0 nosy_names = ['gvanrossum', 'tim.peters', 'fdrake', 'tmick'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue400510' versions = [] ```

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

[freddrake]

This looks good on platforms I have easy access to (linux/x86, linux/ia32 simulator). I do not have Linux+LFS (large file support; a kernel patch) installed on any machines, and could not test in that environment without having a separate machine to use as a test bed.

Accepted as is; we can work out any problems that arise as needed, but it doesn't seem to break things.

[gvanrossum]

Tim- after you're fine with this, please assign to someone with a Linux box for further review.

[gvanrossum]

This was accepted ages ago. What are you waiting for?

[tim-one]

Looks good to me. Passed on to Fred. Fred, Guido wants someone to check this on Linux. Since you have the 64-bit Linux development disk, you're elected \<wink>.

[tim-one]

Fred, do something with this or assign it to some other Unix geek?

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so.

I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation.

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

This patch adds largefile support for Win64 and Linux64 (the latter already motly worked I think), and fixes some possible buffer overflows on all systems with largefile support.

NOTE: this patch depends on my earlier patch to PC/config.h and configure.in for SIZEOF_OFF_T and SIZEOF_FPOS_T. (see: "[Patches] changes to configure.in and PC/config.h for 64-bit systems")

Win64 largefile support involved fixing file_seek, file_tell, and file_truncate to properly handle large indeces and to use the proper Win64 system APIs. Win64 does not have 64-bit capable versions of ftell and fseek, this could be worked around with fgetpos() and fsetpos() (and _telli64(), and 64-bit tell()). _portable_ftell() and _portable_fseek() were written to hold the platform dependent logic. You are still restricted to 32-bits for single reads and writes.

Previously _chsize was used blindly as the replacement for ftruncate() in Win32. In fact, _chsize() is not 64-bit capable so and appropriate overflow check was added.

There are some type histrionics involved because off_t is only 32-bits on Win64. fpos_t is 64-bits, however, so fpos_t is used for Win64.

As well, the patch adds some necessary overflow checks (raising OverflowError when an overflow is detected). See file_read for example.

A forthcoming patch adds a test to the test suite for this largefile support.

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

Sorry about the delay. Had to make some small changes to get the old patch to apply.