Fatal Python error: Interpreter not initialized

[dc82ae88-f879-4394-9ed2-7f3f8b78e210]

BPO	1332869
Nosy	@mwhudson, @tim-one, @vstinner, @ericsnowcurrently
Files	399461.txt: Backtrace collect-fatal.diff: 1

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['interpreter-core', 'type-bug'] title = 'Fatal Python error: Interpreter not initialized' updated_at = user = 'https://bugs.python.org/ajmitch' ``` bugs.python.org fields: ```python activity = actor = 'vstinner' assignee = 'nnorwitz' closed = True closed_date = closer = 'vstinner' components = ['Interpreter Core'] creation = creator = 'ajmitch' dependencies = [] files = ['1820', '1821'] hgrepos = [] issue_num = 1332869 keywords = [] message_count = 11.0 messages = ['26643', '26644', '26645', '26646', '26647', '26648', '26649', '26650', '26651', '114629', '368887'] nosy_count = 6.0 nosy_names = ['mwh', 'tim.peters', 'nnorwitz', 'vstinner', 'ajmitch', 'eric.snow'] pr_nums = [] priority = 'normal' resolution = 'out of date' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue1332869' versions = ['Python 2.6'] ```

[dc82ae88-f879-4394-9ed2-7f3f8b78e210]

When running 'bzr status' on Ubuntu 5.10 with python 2.4.2, I came across the error: Fatal Python error: Interpreter not initialized (version mismatch?) Aborted

The bzr code in question has a __del__ method which when invoked (eventually) imports socket (which imports _socket), causing everything to fall over in a heap.

A backtrace is available at http://pastebin.com/399461

[d21744ff-f396-4c71-955e-7dbd2e886779]

Logged In: YES user_id=33168

Are there any other extension modules in the system that are not provided with python? When you went into gdb did it print a long list of dynamic modules loaded? Can you provide that info too?

I'm guessing this is a memory overwrite which can happen with poorly behaved C extension modules. You could try running python under valgrind and see what it reports.

[d21744ff-f396-4c71-955e-7dbd2e886779]

Logged In: YES user_id=33168

Also, where is the bzr code and what version are you running?

[dc82ae88-f879-4394-9ed2-7f3f8b78e210]

Logged In: YES user_id=19159

Feedback in #bzr indicated that I was triggering this code: http://pastebin.com/399491 and also something similar to http://mail.python.org/pipermail/python-dev/2003-September/038151.html Another example that triggers it is http://mail.python.org/pipermail/python-dev/2003-November/040188.html It's all a stock breezy system, no external modules, and bzr is retrieved from running: rsync -av bazaar-ng.org::bazaar-ng/bzr/bzr.dev . I was using the latest revision at the time this happened.

[dc82ae88-f879-4394-9ed2-7f3f8b78e210]

Logged In: YES user_id=19159

Also, gdb did not report other modules loaded

[mwhudson]

Logged In: YES user_id=6656

Are you using daemon threads?

Why is a __del__ method importing things?

[d21744ff-f396-4c71-955e-7dbd2e886779]

Logged In: YES user_id=33168

Hey Tim, I'm hoping you can take some time to long at this patch (you too Michael and anyone else listening).

Attached is a patch which seems to fix this problem. At least it fixes it from Tim's example in the mail from Nov (I think). It moves initialized down after the GC collect.

Is this patch a good or bad thing?

Andrew, it would be interesting if you could test this patch (you will need to rebuild python) for bzr. I couldn't trigger the problem from my version.

[d21744ff-f396-4c71-955e-7dbd2e886779]

Logged In: YES user_id=33168

Well, what I really hope is that Tim can *make some time to *look at this patch. Feel free to long over it if you wish. :-)

[tim-one]

Logged In: YES user_id=31435

Sorry for the delay! I didn't realize this was assigned to me. Note that the possiblity for problems here was already noted in Py_Finalize():

    /* Collect garbage.  This may call finalizers; it's nice to
call these
     * before all modules are destroyed.
     * XXX If a __del__ or weakref callback is triggered here,
and tries to
     * XXX import a module, bad things can happen, because
Python no
     * XXX longer believes it's initialized.
     * XXX     Fatal Python error: Interpreter not initialized
(version mismatch?)
     * XXX is easy to provoke that way.  I've also seen, e.g.,
     * XXX     Exception exceptions.ImportError: 'No module
named sha'
     * XXX         in <function callback at 0x008F5718> ignored
     * XXX but I'm unclear on exactly how that one happens.  In
any case,
     * XXX I haven't seen a real-life report of either of these.
     */
    PyGC_Collect();

I don't think it would do much harm to move "initialized = 0" down, although it's impossible to get truly concerned about code doing imports in __del when the interpreter is tearing itself down. Note that sys exit funcs have already been called, and signals have been disabled, by this time, so there would still be ways for dubious __del code to fail.

Anyway, if you want to do this, the XXX comments reproduced above should be changed too (e.g., deleted).

Assigning to Neal, so that he can get practice following his new "no changes without a test case" guideline :-)

[83d2e70e-e599-4a04-b820-3814bbdb9bef]

Is this still valid?

[vstinner]

No activity since 2006, I close the issue as out of date.