fix mmap module (broken seek, 64-bit stuff, overflows)

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

BPO	400515
Nosy	@gvanrossum, @tim-one, @akuchling
Files	None: None

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/akuchling' closed_at = created_at = labels = [] title = 'fix mmap module (broken seek, 64-bit stuff, overflows)' updated_at = user = 'https://bugs.python.org/tmick' ``` bugs.python.org fields: ```python activity = actor = 'tim.peters' assignee = 'akuchling' closed = True closed_date = None closer = None components = ['None'] creation = creator = 'tmick' dependencies = [] files = ['2446'] hgrepos = [] issue_num = 400515 keywords = ['patch'] message_count = 7.0 messages = ['32743', '32744', '32745', '32746', '32747', '32748', '32749'] nosy_count = 4.0 nosy_names = ['gvanrossum', 'tim.peters', 'akuchling', 'tmick'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue400515' versions = [] ```

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

[gvanrossum]

Andrew, I'm not sure if you can review patches for Windows, but since it's your module, I'd like you to have a look at this first. Tim should get it next.

[akuchling]

The patches look OK, particularly the fixing of the seek() method. I'm not sure why various casts are changed from (long) to (int), and wonder if those casts could simply be removed, but that's not a problem.

[tim-one]

Best I can tell, this has already been checked in! So marking Closed, and assigning back to AMK. Andrew, scream at me if I done wrong here.

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so.

I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation.

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

This patch fixes some issues in the mmap module. The changes are:

• The use of HFILE for Win32 is no longer appropriate for Win32/64, all of the associated system calls use a pointer length integer, hence HFILE is replaced with INT_PTR (a Windows world typedef for this)

• Do proper bounds checking limiting the length of a mmap'd file to [0, INT_MAX] (see _GetMapSize()). INT_MAX is chosen because this is the current restriction on the length of Python objects and a mmap'd file larger than this cannot be used effectively anyway. For instance a mmap'd file longer than INT_MAX cannot be sliced beyond INT_MAX, or indexed, etc. Until the limit on the size of Python objects increases then there is little use in having larger mmap'd files.

• The mmap_seek is fairly significantly broken (seeking from the current position or from the end are both borken, IIRC). This patch fixes that, checks for possible out of range seek values, and extends test_mmap.py to test this stuff.

• Use a C int for variables associated with the length field of Python objects, this is what it is and the distinction matters on Linux64, for example.

• Use safe number literals. (some unsigned)0xFFFFFFFF -> (some unsigned)-1 The former does not give the intended result iff 'some unsigned' is > 32-bits.

[bdaf8532-ded6-4d4c-9bb3-51fd76040dc1]

Some of the casts were changed from (long) to (int) because 'int' was the size being returned (for example in the length value to PyString_FromStringAndSize, or the mmap find method is specified to return an int and not a long, note that sizeof(int) != sizeof(long) on Linux64.

Note that this has already been checked in: http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/dist/src/Modules/mmapmodule.c.diff?cvsroot=python&r1=2.12&r2=2.13&f=H

...and in the actual checkin the first cast to (int) was dropped anyway :). Maybe you did that yourself, Andrew.