python / cpython

The Python programming language
https://www.python.org
Other
62.36k stars 29.95k forks source link

subprocess isn't friendly to other Python implementations with different GCs #54023

Open DinoV opened 14 years ago

DinoV commented 14 years ago
BPO 9814
Nosy @merwok, @DinoV

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'library'] title = "subprocess isn't friendly to other Python implementations with different GCs" updated_at = user = 'https://github.com/DinoV' ``` bugs.python.org fields: ```python activity = actor = 'christian.heimes' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'dino.viehland' dependencies = [] files = [] hgrepos = [] issue_num = 9814 keywords = [] message_count = 2.0 messages = ['115979', '116131'] nosy_count = 3.0 nosy_names = ['astrand', 'eric.araujo', 'dino.viehland'] pr_nums = [] priority = 'low' resolution = None stage = 'needs patch' status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue9814' versions = ['Python 2.7', 'Python 3.3', 'Python 3.4'] ```

DinoV commented 14 years ago

subprocess isn't very friendly to implementations with different GCs then CPython and in general relies on behavior that's not going to be very stable. I noticed the following issues but was able to work around all of them.

First Popen is a finalizable object which holds onto another finalizable object (the process handle). In __del__ Popen calls internal poll which attempts to wait on the handle. But because both the handle and the POpen object can go out of scope at the same time the handle can be finalized first. The end result is that you cannot get the return code and therefore the Popen class will resurrect it's self. That will at the very least leak the Popen object and when running the subprocess tests will eventually start failing all of the tests. For me this was happening because test_cwd doesn't call wait and therefore never explicitly gets the exit code.

Resurrecting finalizable objects seems like a pretty bad idea and seems to be there only for test purposes - after all who cares if you collect the exit code after no one refers to the object. Unless _active is supposed to be publicly exposed?

Another issue is relying on CPython's reference counting garbage collcetion. In particular _get_handles on Windows does "p2cread = self._make_inheritable(p2cread)" where stdin == PIPE. On CPython this is going to remove the 1 reference to the previous p2cread and close it immediately. With other GCs this reference will linger around only to be closed at some point. Usually that's not a problem but because this handle remains in the parent process the pipe does not get closed when it should resulting in hangs. This is effectively a duplicated handle of the handle _execute_child closes after the child is launched.

merwok commented 14 years ago

I think all developers agree that we should play nice with all VMs. Do you want to make patches for these problems?