search

tkf / emacs-jedi

Python auto-completion for Emacs
http://tkf.github.io/emacs-jedi/latest/
667 stars 89 forks source link

Kernel Panic in Mac OS X #37

Closed asmeurer closed 11 years ago

asmeurer commented 11 years ago

Today, I just installed emacs-jedi, as well as all of its dependencies (except for auto-complete, which I already had installed). I installed everything from git. I then enabled everything using https://github.com/asmeurer/dotfiles/blob/master/.emacs#L759-785 (except with the last part uncommented.

I then tried editing some Python. It worked for a little bit, but then my system kernel panicked! Here is the log

Interval Since Last Panic Report:  191924 sec
Panics Since Last Report:          2
Anonymous UUID:                    563FC21E-12C7-8C2A-E396-225C427EB6F4

Tue Mar 19 17:25:59 2013
panic(cpu 6 caller 0xffffff800191edba): "negative open count (c, 16, 6)"@/SourceCache/xnu/xnu-2050.22.13/bsd/miscfs/specfs/spec_vnops.c:1813
Backtrace (CPU 6), Frame : Return Address
0xffffff8142cabbe0 : 0xffffff800181d626 
0xffffff8142cabc50 : 0xffffff800191edba 
0xffffff8142cabc90 : 0xffffff8001923c46 
0xffffff8142cabce0 : 0xffffff8001910cb6 
0xffffff8142cabd20 : 0xffffff80019084cf 
0xffffff8142cabd50 : 0xffffff8001907665 
0xffffff8142cabda0 : 0xffffff8001b4b60c 
0xffffff8142cabe10 : 0xffffff8001b4bb86 
0xffffff8142cabe50 : 0xffffff8001b55a1b 
0xffffff8142cabec0 : 0xffffff8001839ce9 
0xffffff8142cabef0 : 0xffffff800183c7e8 
0xffffff8142cabf20 : 0xffffff800183c65e 
0xffffff8142cabf50 : 0xffffff800181b70d 
0xffffff8142cabf90 : 0xffffff80018b84a3 
0xffffff8142cabfb0 : 0xffffff80018cd4ac 

BSD process name corresponding to current thread: python

Mac OS version:
12D78

Kernel version:
Darwin Kernel Version 12.3.0: Sun Jan  6 22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64
Kernel UUID: 3EB7D8A7-C2D3-32EC-80F4-AB37D61492C6
Kernel slide:     0x0000000001600000
Kernel text base: 0xffffff8001800000
System model name: MacBookPro10,1 (Mac-C3EC7CD22292981F)

System uptime in nanoseconds: 550977231116
last loaded kext at 244798419625: com.apple.filesystems.msdosfs 1.8 (addr 0xffffff7f82508000, size 65536)
last unloaded kext at 447749174583: com.apple.iokit.IOEthernetAVBController 1.0.2b1 (addr 0xffffff7f829df000, size 28672)
loaded kexts:
org.pqrs.driver.KeyRemap4MacBook    8.0.38
org.virtualbox.kext.VBoxNetAdp  4.2.1
org.virtualbox.kext.VBoxNetFlt  4.2.1
org.virtualbox.kext.VBoxUSB 4.2.1
org.virtualbox.kext.VBoxDrv 4.2.1
com.apple.driver.AppleHWSensor  1.9.5d0
com.apple.driver.AudioAUUC  1.60
com.apple.filesystems.autofs    3.0
com.apple.iokit.IOBluetoothSerialManager    4.1.3f3
com.apple.driver.AGPM   100.12.87
com.apple.driver.ApplePlatformEnabler   2.0.6d1
com.apple.driver.X86PlatformShim    1.0.0
com.apple.driver.AppleMikeyHIDDriver    122
com.apple.driver.AppleHDA   2.3.7fc4
com.apple.driver.AppleUpstreamUserClient    3.5.10
com.apple.driver.AppleIntelHD4000Graphics   8.1.0
com.apple.GeForce   8.1.0
com.apple.iokit.BroadcomBluetoothHCIControllerUSBTransport  4.1.3f3
com.apple.driver.AppleMikeyDriver   2.3.7fc4
com.apple.iokit.IOUserEthernet  1.0.0d1
com.apple.Dont_Steal_Mac_OS_X   7.0.0
com.apple.driver.AppleMuxControl    3.3.0
com.apple.driver.AppleSMCPDRC   1.0.0
com.apple.driver.AppleSMCLMU    2.0.3d0
com.apple.driver.AppleLPC   1.6.0
com.apple.driver.ApplePolicyControl 3.3.0
com.apple.driver.AppleMCCSControl   1.1.11
com.apple.driver.AppleIntelFramebufferCapri 8.1.0
com.apple.driver.AppleUSBTCButtons  237.1
com.apple.driver.AppleUSBTCKeyboard 237.1
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 34
com.apple.driver.XsanFilter 404
com.apple.iokit.IOAHCIBlockStorage  2.3.1
com.apple.driver.AppleUSBHub    5.5.5
com.apple.driver.AppleSDXC  1.4.0
com.apple.driver.AirPort.Brcm4331   614.20.16
com.apple.driver.AppleAHCIPort  2.5.1
com.apple.driver.AppleUSBEHCI   5.5.0
com.apple.driver.AppleUSBXHCI   5.5.5
com.apple.driver.AppleEFINVRAM  1.7
com.apple.driver.AppleSmartBatteryManager   161.0.0
com.apple.driver.AppleACPIButtons   1.7
com.apple.driver.AppleRTC   1.5
com.apple.driver.AppleHPET  1.8
com.apple.driver.AppleSMBIOS    1.9
com.apple.driver.AppleACPIEC    1.7
com.apple.driver.AppleAPIC  1.6
com.apple.driver.AppleIntelCPUPowerManagementClient 196.0.0
com.apple.nke.applicationfirewall   4.0.39
com.apple.security.quarantine   2
com.apple.driver.AppleIntelCPUPowerManagement   196.0.0
com.apple.kext.triggers 1.0
com.apple.iokit.IOSCSIArchitectureModelFamily   3.5.5
com.apple.iokit.IOSerialFamily  10.0.6
com.apple.driver.DspFuncLib 2.3.7fc4
com.apple.iokit.IOAudioFamily   1.8.9fc11
com.apple.kext.OSvKernDSPLib    1.6
com.apple.iokit.IOAcceleratorFamily 30.14
com.apple.nvidia.gk100hal   8.1.0
com.apple.NVDAResman    8.1.0
com.apple.iokit.AppleBluetoothHCIControllerUSBTransport 4.1.3f3
com.apple.iokit.IOSurface   86.0.4
com.apple.iokit.IOBluetoothFamily   4.1.3f3
com.apple.driver.AppleBacklightExpert   1.0.4
com.apple.driver.AppleHDAController 2.3.7fc4
com.apple.iokit.IOHDAFamily 2.3.7fc4
com.apple.driver.AppleSMBusPCI  1.0.11d0
com.apple.driver.AppleGraphicsControl   3.3.0
com.apple.iokit.IONDRVSupport   2.3.7
com.apple.driver.X86PlatformPlugin  1.0.0
com.apple.driver.AppleSMC   3.1.4d2
com.apple.driver.IOPlatformPluginFamily 5.3.0d51
com.apple.driver.AppleSMBusController   1.0.11d0
com.apple.iokit.IOGraphicsFamily    2.3.7
com.apple.driver.AppleUSBMultitouch 237.3
com.apple.iokit.IOUSBHIDDriver  5.2.5
com.apple.driver.AppleThunderboltDPInAdapter    1.8.9
com.apple.driver.AppleThunderboltDPAdapterFamily    1.8.9
com.apple.driver.AppleThunderboltPCIDownAdapter 1.2.6
com.apple.driver.AppleUSBMergeNub   5.5.5
com.apple.driver.AppleUSBComposite  5.2.5
com.apple.driver.AppleThunderboltNHI    1.6.3
com.apple.iokit.IOThunderboltFamily 2.2.6
com.apple.iokit.IOUSBUserClient 5.5.5
com.apple.iokit.IO80211Family   522.4
com.apple.iokit.IONetworkingFamily  3.0
com.apple.iokit.IOAHCIFamily    2.3.1
com.apple.iokit.IOUSBFamily 5.5.5
com.apple.iokit.IOHIDFamily 1.8.1
com.apple.driver.AppleEFIRuntime    1.7
com.apple.iokit.IOSMBusFamily   1.1
com.apple.security.sandbox  220.2
com.apple.kext.AppleMatch   1.0.0d1
com.apple.security.TMSafetyNet  7
com.apple.driver.DiskImages 345
com.apple.iokit.IOStorageFamily 1.8
com.apple.driver.AppleKeyStore  28.21
com.apple.driver.AppleACPIPlatform  1.7
com.apple.iokit.IOPCIFamily 2.7.3
com.apple.iokit.IOACPIFamily    1.4
com.apple.kec.corecrypto    1.0
Model: MacBookPro10,1, BootROM MBP101.00EE.B02, 4 processors, Intel Core i7, 2.3 GHz, 8 GB, SMC 2.3f35
Graphics: Intel HD Graphics 4000, Intel HD Graphics 4000, Built-In, 512 MB
Graphics: NVIDIA GeForce GT 650M, NVIDIA GeForce GT 650M, PCIe, 1024 MB
Memory Module: BANK 0/DIMM0, 4 GB, DDR3, 1600 MHz, 0x80AD, 0x484D54333531533642465238432D50422020
Memory Module: BANK 1/DIMM0, 4 GB, DDR3, 1600 MHz, 0x80AD, 0x484D54333531533642465238432D50422020
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0xEF), Broadcom BCM43xx 1.0 (5.106.98.100.16)
Bluetooth: Version 4.1.3f3 11349, 2 service, 11 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en0
Serial ATA Device: APPLE SSD SM256E, 251 GB
USB Device: hub_device, 0x8087  (Intel Corporation), 0x0024, 0x1a100000 / 2
USB Device: FaceTime HD Camera (Built-in), apple_vendor_id, 0x8510, 0x1a110000 / 3
USB Device: hub_device, 0x8087  (Intel Corporation), 0x0024, 0x1d100000 / 2
USB Device: hub_device, 0x0424  (SMSC), 0x2512, 0x1d180000 / 3
USB Device: BRCM20702 Hub, 0x0a5c  (Broadcom Corp.), 0x4500, 0x1d181000 / 5
USB Device: Bluetooth USB Host Controller, apple_vendor_id, 0x8286, 0x1d181300 / 8
USB Device: Apple Internal Keyboard / Trackpad, apple_vendor_id, 0x0262, 0x1d182000 / 4

This is in Mac OS X 10.8.3. I'm using a git version of emacs compiled in January, and iTerm2. I tried it a second time, and it panicked again (same log; I can paste them both if you want).

I kind of doubt this is the fault of emacs-jedi, but I was wondering if you could help point me in the right direction of who to report this to get it fixed (or at least worked around)?

tkf commented 11 years ago

I have no idea, sorry. A few ideas to investigate what is wrong:

  1. Run make test. You need carton for that. Install it, or specify the path like this: make CARTON=PATH/TO/carton/bin/carton test

  2. Try with minimal .emacs. You can save the lines you specified in your .emacs to a different file, and then start Emacs by emacs -Q -l PATH/TO/THE/FILE.el.

    Alternatively, you can do make tryout.

  3. You can also run tests for Python-EPC like this: nosetests epc, though I don't know this is relevant.
asmeurer commented 11 years ago

Is there some other dependency for the tests? I get

Warning: Lisp directory `/Users/aaronmeurer/Documents/emacs-jedi/elpa/install.log': Not a directory
Cannot open load file: mocker
make[1]: *** [test-1] Error 255
make: *** [test] Error 2
tkf commented 11 years ago

If you have carton then it should run without problem. It runs on Travis always like that, though I've never run it on Mac OS.

What is in elpa/? Can you do something like tree elpa or find elpa? Also please paste what is in elpa/install.log.

asmeurer commented 11 years ago

make tryout didn't work, but I was still able to reproduce with this minimal startup file. The computer panics when I try to exit emacs.

asmeurer commented 11 years ago

Noting in elpa except for elpa/archives/gnu/archive-contents (this is ~/.emacs.d/elpa).

tkf commented 11 years ago

Are these libraries loaded from test_emacs.el are compiled? Did you try removing all *.elc files?

tkf commented 11 years ago

I mean elpa under emacs-jedi directory.

asmeurer commented 11 years ago

Oh, duh, you're talking about the elpa referenced by the log. That was because I didn't put the full path to carton the first time.

asmeurer commented 11 years ago

Removed elc files. Still panics. According to the panic log, it is python, not emacs, that is causing the panic.

tkf commented 11 years ago

So how about testing python-epc?

asmeurer commented 11 years ago 
$nosetests2.7 epc
.....................S....SERROR:epc:EPCClosed()
..................................
----------------------------------------------------------------------
Ran 61 tests in 25.880s

OK (SKIP=2)
tkf commented 11 years ago

The error message showed up is actually OK... So probably run Jedi test? It is not distributed with the module, so you need to get it from github https://github.com/davidhalter/jedi.

tkf commented 11 years ago

Ah, I forgot that we can run Jedi server in debugging mode. Try jedi:toggle-debug-server command: http://tkf.github.com/emacs-jedi/#jedi:toggle-debug-server

You can omit --pdb for just seeing the traceback (if Python could print out something useful).

asmeurer commented 11 years ago

Jedi tests pass just fine. I'm going to wait to test the server until my computer is not doing anything, so that a panic is not too interrupting.

asmeurer commented 11 years ago

It doesn't seem to panic in debug mode.

By the way, it also never seems to actually work, though I'm not positive. Is there a minimal way to test that it's using Jedi and not just auto-complete?

asmeurer commented 11 years ago

Assumedly debug mode doesn't do it because it happens when I exit emacs, and this doesn't stop the server in debug mode.

tkf commented 11 years ago

Try M-x jedi:show-jedi-version. You don't even need to start python-mode.

Also, checkout options of the server. You can increase logging level.

asmeurer commented 11 years ago

I have no doubt that Jedi is running, but how do I check that it is actually working?

By the way, just running show version panics on exit.

tkf commented 11 years ago

If jedi:show-jedi-version pops up a buffer with version info, that means server is working. This command calls a remote command on the server.

panics on exit

Is it always on exit? What happen if you kill all servers before exit? You can do that by M-x epc:controller then hit K for each rows.

asmeurer commented 11 years ago

Yes, the servers are running, but I believe that the actual completion isn't. It's hard to tell, though, because auto-complete does work.

I'll try the increased logging and killing the servers next time my computer isn't doing something.

dliappis commented 11 years ago

Same problem here.

I use emacs-24.3 installed from macports and all the rope/ropemacs/ropemode/pymacs/autocomplete stuff.

As soon as I start emacs and start editing a python file, after having performed some pymacs related thing e.g. completion, when I save/exit (C-x S C) OS reboots unexpectedly and on restart I see:

panic(cpu 6 caller 0xffffff800191edba): "negative open count (c, 16, 6)"@/SourceCache/xnu/xnu-2050.22.13/bsd/miscfs/specfs/spec_vnops.c:1813 Backtrace (CPU 6), Frame : Return Address

I've been running emacs macport version on a company laptop (with Lion 10.7.x) and I never faced this issue.

The version details:

Running on Mountain Lion 10.8.3, using macports 2.1.3 and the following macports software versions:

python 2.7.3 emacs 24.3

Here are the relevant package versions installed with pip:

Pymacs==0.25 rope==0.9.4 ropemacs==0.7 ropemode==0.2

Here are the extra things installed under site-lisp:

$ ls -l /opt/local/share/emacs/site-lisp/ | awk '{print $9}'

doctest-mode.el doctest-mode.elc gnuplot-eldoc.el gnuplot-eldoc.elc idna.el magit-bisect.el magit-bisect.elc magit-blame.el magit-blame.elc magit-key-mode.el magit-key-mode.elc magit-stgit.el magit-stgit.elc magit-svn.el magit-svn.elc magit-topgit.el magit-topgit.elc magit-wip.el magit-wip.elc magit.el magit.elc punycode.el pycomplete.el python-mode.el python-mode.elc rebase-mode.el rebase-mode.elc subdirs.el

And the .emacs file:

(add-to-list 'load-path "~/.emacs.d/auto-complete") (add-to-list 'load-path "/opt/local/share/emacs/site-lisp") (autoload 'python-mode "python-mode" "Python Mode." t) (add-to-list 'auto-mode-alist '(".py\'" . python-mode)) (add-to-list 'interpreter-mode-alist '("python" . python-mode)) (require 'python-mode)

(autoload 'pymacs-apply "pymacs") (autoload 'pymacs-call "pymacs") (autoload 'pymacs-eval "pymacs" nil t) (autoload 'pymacs-exec "pymacs" nil t) (autoload 'pymacs-load "pymacs" nil t) (autoload 'pymacs-autoload "pymacs")

(pymacs-load "ropemacs" "rope-") (setq ropemacs-enable-autoimport t)

(require 'auto-complete) (global-auto-complete-mode t)

tkf commented 11 years ago

@dliappis You don't use jedi.el, right? This makes me think that it may be nothing to do with my problem but Emacs launching subproces in Mac OS X.

Does the same thing happen if you do something like M-! ls RET? Or probably this is needed to be a long lasting process, like M-! sleep 5m & RET?

dliappis commented 11 years ago

@tkf Nope I don't use jedi.el.

I am certain this is totally irrelevant to emacs because I tried using aquamacs instead and after a couple of minutes of fighting around with some source code, I got the same crash.

The commands you mentioned above do not reproduce the problem.

However the problem is always reproducible if pymacs is loaded and I try autocomplete on a class and then try to save/exit emacs.

This problem seems to be somehow related to python (in particular the macports version of python27)

Which version of python are you using?

tkf commented 11 years ago

I am not using Mac OS so I guess @asmeurer's python version is more interesting.

If it is Python, how about M-! python -c "print 'hello'" RET. You can put sleep or some intense computation in there to check it it crashes your machine. Probably you need to read and write to/from pipe or socket.

dliappis commented 11 years ago

Here's some more investigation:

Since I was suspecting python to be a problem, I tried the default python version (2.7.2) installed with OSX Mountain Lion.

I used emacs 24.3 (from macports) and also Aquamacs (based on emacs 23) and in both cases I got the same reboot/crash problem!

The test case is always the same: Edit a python file, try auto-complete for a couple of classes, save the file, exit emacs -> crash happens when exiting emacs.

I even ditched macports and tried homebrew (an alternative package management system for OSX). I installed python 2.7.3 and emacs 24.3 using brew, installed pymacs, python-mode and the rest of the dependencies, leading to the same problem.

I am totally baffled; is there an issue with Mountain Lion, when emacs is launching python processes?

I tried your simple:

M-! python -c "print 'hello'" RET

test which was successful. As you suggested I will try a more intense python computation in the background.

I think this is perhaps the time to raise a bug with Apple?

asmeurer commented 11 years ago

I am using fink python 2.7.

asmeurer commented 11 years ago

And yes, this is obviously an apple bug. A kernel panic is by definition a bug in e kernel (unless kernel extensions are involved or there is a hardware issue, which are clearly not the case here). The xnu/.../spec_vnops.c is part of the Mac OS X kernel (xnu is the OS X kernel). Unfortunately, they only release source code for 10.7, and it doesn't even have 1813 lines.

I was just hoping to find a workaround here.

asmeurer commented 11 years ago

Actually @dliappis what kind of computer do you have? Mine is a 15" retina MacBook Pro.

dliappis commented 11 years ago

@asmeurer I am using a MacBook Air Mid 2012.

I am also desperately trying to find a workaround as well, as I can't imagine having to use another editor (used to emacs for too long.)

I have opened a bug in Apple's bug tracking system (13496168); hoping for a quick resolution -- perhaps if you open a case as well they could speed up resolution/increase priority?

As mentioned above any combination I've tried with emacs and python has resulted to the same reboot!

asmeurer commented 11 years ago

Yes, I will open a bug as well. I remember hearing that Apple considers multiple reports as "votes", so each person who sees this should report the bug. We should also probably open an issue on open radar. And obviously, when it happens, click the report button for the crash report.

dliappis commented 11 years ago

Yes, I agree on all points. I am hoping this will be dealt swiftly as a kernel panic is a major issue. This sure is Mountain Lion specific, as my work laptop (Macbook pro 13" with Lion) with similar python+emacs setup has been super stable for a long time.

tkf commented 11 years ago

Probably you can workaround the bug by using different Python version like 3.2 until Apple fix the bug?

asmeurer commented 11 years ago

How do you switch which Python executable is used?

Also, do Jedi and all the other dependencies work in Python 3? Will it expect the Python sources it analyzes to be Python 3?

tkf commented 11 years ago

Python modules work with 3.2. Jedi running with Python 3 can parse Python 2 code. You need to install modules for Python 3 if you want to complete functions in them, though.

To use Python 3, do make PYTHON=python3 clean requirements

asmeurer commented 11 years ago

OK, some more testing:

asmeurer commented 11 years ago

Even if I force quit iterm 2 (command-option-esc), it panics.

dliappis commented 11 years ago

It seems that bug is a known one, but so far I don't see any updates on Mountain Lion ... I am really hoping Apple will sort this one out, as it's literally made emacs unusable for python.

sbrother commented 11 years ago

This is happening to me too. I hope they sort it out, because my whole workflow is based on Python in emacs. Works fine in Linux but I have to use a Mac for my new job. Has anyone raised a bug with Apple?

asmeurer commented 11 years ago

@sbrother are you getting it from Jedi or pymacs (or something else)?

As noted above, an issue has been opened, but apple counts multiple reports as "votes" so you should submit it again (and also send them your crash reports when your computer reboots).

sbrother commented 11 years ago

I have ropemacs, autocorrect, rope-mode, and pymacs running, so I assume it's related to pymacs? I just submitted a bug report to Apple.

asmeurer commented 11 years ago

It's something to do with emacs and Python in general. These are the libraries that do it for me, which are just Jedi and its (elisp only) dependencies. Does pymacs also use EPC?

asmeurer commented 11 years ago

I opened this as bug 13615108. I also put it on open radar.

tkf commented 11 years ago

Does pymacs also use EPC?

No. Pymacs and EPC are different ways of inter-process communication. The similarity I see is that both launch Python process from Emacs.

tkf commented 11 years ago

Come to think of it, this may be the reason why there is no panic in the debug mode. In the debug mode, Python process is launched outside of Emacs. If you remove certain flags such as --pdb, I think launching EPC server this way is usable.

asmeurer commented 11 years ago

As I noted, it also doesn't panic if I manually quit it in the EPC controller.

Also as I noted, am not really sure if this is even working at all in any case (but again without a reliable test case I can't be sure). Assumedly something like 

import sys
sys.TAB

should complete the functions in the sys module, no? If so, then it isn't working.

tkf commented 11 years ago

Did you run it w/o --pdb? If you give --pdb, it may hang due to some error. If you don't give it the error is just ignored.

If quitting from EPC controller is fine, then adding a elisp function to kill all EPC processes to kill-emacs-hook should solve this problem.

tkf commented 11 years ago

@asmeurer This should be equivalent to "kill all Jedi EPC servers before kill Emacs". This probably makes emacs-jedi usable for Mac users.

(defun jedi:stop-all-servers ()
  (maphash (lambda (_ mngr) (epc:stop-epc mngr))
           jedi:server-pool--table))

(add-hook 'kill-emacs-hook #'jedi:stop-all-servers)
asmeurer commented 11 years ago

Ok, I'll test it when I get the nerves to possibly panic again. Even if this works, it will be living on the edge, because as I noted, if I ever have to kill -9 emacs or force quit the terminal or terminal tab, it will panic.

asmeurer commented 11 years ago

OK, I think that "works," in the sense that it doesn't panic (yay!). But as I noted before, I'm pretty sure that jedi is not working at all. My completions don't seem to be any better than what I get with just auto-complete. I'll open a new issue for it.

tkf commented 11 years ago

@asmeurer If you need to kill -9, probably killing subprocesses before that helps... You can use Emacs's daemon mode to save it from accidentally closing tabs.