Modify co_filename in frozen programs

[8b9e7e07-4219-4888-b494-3b044b438c92]

BPO	401702
Nosy	@gvanrossum
Files	None: None

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/gvanrossum' closed_at = created_at = labels = [] title = 'Modify co_filename in frozen programs' updated_at = user = 'https://bugs.python.org/lhudson' ``` bugs.python.org fields: ```python activity = actor = 'gvanrossum' assignee = 'gvanrossum' closed = True closed_date = None closer = None components = ['Demos and Tools'] creation = creator = 'lhudson' dependencies = [] files = ['2845'] hgrepos = [] issue_num = 401702 keywords = ['patch'] message_count = 12.0 messages = ['34502', '34503', '34504', '34505', '34506', '34507', '34508', '34509', '34510', '34511', '34512', '34513'] nosy_count = 3.0 nosy_names = ['gvanrossum', 'jhylton', 'lhudson'] pr_nums = [] priority = 'normal' resolution = 'accepted' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue401702' versions = [] ```

[8b9e7e07-4219-4888-b494-3b044b438c92]

[gvanrossum]

I like the idea, but I have a few suggestions.

• To avoid the case sensitivity issue you mention, you could use os.path.normpath() before the string comparisons.

• Rather than writing to a real temporary file, you could marshal to a string and than unmarshal from a StringIO file.

• And the big whopper: rather than using a modified version of unmarshal, I suggest writing a recursive code object transformer. It takes a code object and a replace_paths list, and returns a similar code object with the co_filename attribute changed according to the list. The trick is that it should also do this, recursively, to any code object found in the co_consts tuple. (That's the only place where code objects can occur recursively.) While this is probably not much less code than the marshaller you wrote, it has the advantage of not being dependent on the details of the marshal format. Please upload a new patch -- I'm interested in getting this into Python 2.1.

[gvanrossum]

Thanks for the new patch! I'll assign this to Jeremy for further review and to carry it to completion. (Jeremy, if you're swamped, please find someone else to do it.)

Re normpath: my mistake -- I meant normcase.

[gvanrossum]

I'm taking this, on the assumption that Mark's too busy and freeze is our shared responsibility.

[gvanrossum]

No time to review this before the 2.1a1 release; I'll do this for 2.1a2.

[03bde425-37ce-4291-88bd-d6cecc46a30e]

This sounds like a reasonable enough feature, but we are in feature freeze for Python 2.0.

[03bde425-37ce-4291-88bd-d6cecc46a30e]

(somewhat) randomly reassigned; don't have time to look at this

[8b9e7e07-4219-4888-b494-3b044b438c92]

A new feature for freeze.

This patch was developed primarily to reduce the size of the frozen binary. It is particularly useful when freezing for 'small' platforms, such as Palm OS, where you really want to save that last miserable byte.

A limitation of this patch is that it does not provide any feedback about the replacements being made. As the path matching is case-sensitive this may lead to unexpected behaviour for DOS and Windows people, eg > freeze.py -r C:\Python\Lib\=py\ goats.py should probably be: > freeze.py -r c:\python\lib\=py\ goats.py

[8b9e7e07-4219-4888-b494-3b044b438c92]

A recursive code object transformer has been implemented as a member of the ModuleFinder.

Case sensitivity issue:
normpath strips trailing '/' or '\\' which reduces the flexibility of the 
path replacement.  It also does not seem to solve the case-sensitivity problem:
    D:\>python
    Python 2.0 (#8, Oct 17 2000, 15:27:24) [MSC 32 bit (Intel)] on win32
    Type "copyright", "credits" or "license" for more information.
    >>> import os
    >>> os.path.normpath("c:\\python\\lib\\")
    'c:\\python\\lib'
    >>> os.path.normpath("c:\\python\\Lib\\")
    'c:\\python\\Lib'
    >>> 
ModuleFinder's debugging system is used to report the status of each unique path encountered.

Temporary file: marshal.dump() and marshal.load() only work with file objects, not file-like objects. Now that new code objects are being created there is no longer any reason to re-marshal so this problem disappears.

[gvanrossum]

Logged In: YES user_id=6380

Shit. Missed the deadline again. Will try before b2... :-(

[gvanrossum]

Logged In: YES user_id=6380

Sure, I see no problem with this patch. I'll see if I can check it in.

If someone else who likes to hack freeze would have it, please go ahead!

[gvanrossum]

Logged In: YES user_id=6380

Applied. Closing now.