YCM Slows down vim startup by ~3 seconds

[jgkamat]

Before I installed YCM, vim started up instantly, but after it causes a very annoying lag whenever I open a new file. Is there a way to keep the YCM server running in the background, or start the server in parallel so I can start editing while YCM is loading?

[agauniyal]

Does this for me while looking for ycm_conf.py (or something like that) file , and then pops a red message that it couldn't find it. As for when I have already specified it , returns to normal speed.

[vheon]

The fact that it lag the startup of vim is a known issue #893 and there there isn't much that we could do about it. The strange thing is that you're experiencing lag on new files. Could you profile it to see if it really is YCM causing that?

[jgkamat]

I'm pretty sure its YCM as when I comment out only the YCM part of my vundle setup, the problem goes away... Is there a way to run the YCM server in the background? Wouldn't that fix the issue?

[oblitum]

Does this for me while looking for ycm_conf.py (or something like that) file , and then pops a red message that it couldn't find it. As for when I have already specified it , returns to normal speed.

This hints not at server issue but maybe an slowndown in the ycm_extra_conf.py search process? (just highlighting...)

[vheon]

Is there a way to run the YCM server in the background?

The server runs in a separated process already. The startup time is spent mostly evaluating the client code. You can profile it to verify.

[jgkamat]

here is my profile of startup time. I ran this with no file arguments. I'm not sure if this is YCM, but this is what happens when I comment out YCM in vundle

[vheon]

We can't see which function is taking the cpu cicles thought. try this:

vim --cmd 'profile start vim.profile' --cmd 'profile func *' --cmd 'profile file *'

Once that vim start in vim use this command:

:profdel func *
:profdel file *

this would produce a file called vim.profile in the directory where you launched vim, inside you can see which function take the most of your cpu during the startup and which line inside the function.

[jgkamat]

The profile file says that 'youcompleteme#Enable()' is taking 1.1 seconds, and 25_SetupUpPython() is taking another 1.1 seconds (its problematic command is 'py from ycm.youcompleteme import YouCompleteMe'). Here is the full profile, no additional addons in vundle. The data I quoted is at the very bottom.

Maybe this is an issue with vundle? Should I try installing YCM with pathogen or manually?

[vheon]

So this is exactly the same results that I got when I profiled some times ago. Look #893. There isn't much we can do in that case. The time is spent loading YCM, plus the caches are probably cold. Closing since is a duplicate issue.

[Aneurin]

I imagine that there are several different issues that could cause these symptoms, but I found for me that it was traced back to the issue described at https://github.com/pyca/pyopenssl/issues/137.

Updating "python-cryptography" to 1.0 reduced the time for SetUpPython() from 6 seconds to 0.6 seconds for me. Anyone on Debian jessie in particular likely has the problematic version of python-cryptography.

[vheon]

We don't use that library directly, don't we?

[Aneurin]

Not directly, no. IIRC it gets imported by pyopenssl which gets imported by requests, which is used in base_request.py: it was the "import requests" line that accounted for almost all of the YCM loading time for me.

[vheon]

So hypothetically if we put the pyopenssl v1.0 into the third_party library as a submodule and put it first in the path then we would load that and potentially improve the startup time? If I do something like pip install pyopenssl can I get the same benefit??

[Aneurin]

This would indicate that the answer is probably yes: http://stackoverflow.com/questions/30708967/python-requests-just-got-very-slow-better-alternative

Note that it is specifically the cryptography module with the issue, even though they didn't dig that deep in that SO thread - I expect that the pip installation described will have pulled in the most recent version as a dependency.

Here is another reference to the bug, which is what led me to the pyopenssl issue and is sort of a bridge to that from the SO question: https://github.com/kennethreitz/requests/issues/2372

Quite possibly requests, pyopenssl, and cryptography will all need to be upgraded in lockstep; personally I did it by installing the version of requests from [Debian] unstable along with all its dependencies, in order to make sure everything in the chain was up to date.

A simple test for the problem is time python -c "import OpenSSL": if it takes more than a second, you have the bad version of cryptography - although to get a difference as dramatic as my 6s -> 0.6s you would also need to be running on a 5 year old Atom.

Sorry I can't be more directly helpful - it was a process of trial and error that took a while before I managed to find the problem, and I didn't make good notes on what I found.

[Valloric]

@Aneurin Thanks for the detective work!

@vheon I'm not a fan of us mucking around with the internals of libraries we depend on. Bad precedent and slippery slope.

If this can be resolved upstream, then that's the way to go.

[vheon]

@Valloric I was thinking more to a note in the README saying that the user could get startup time improvements by installing the new version of pyopenssl by hand.

[Valloric]

@vheon As an entry in the FAQ, sure... with a note saying that this would be an unsupported, best-effort configuration.

[rkiyanchuk]

Thank you @vheon and @Valloric for this investigation.

For the record: I also repeated the profiling on Debian Jessie and got the same results — slow import of pyOpenSSL on version 0.14 (default in Debian packages).

The easiest and safest workaround I came up with was to do user install of latest pyOpenSSL to avoid breaking system-wide packages:

pip install --user --upgrade pyOpenSSL

pyOpenSSL of version 0.15.1 solved slow YCM startup issue for me.

[Herz3h]

I have this slow startup aswell, after profiling, it shows that SetUpPython() takes the longest time during startup.

I have tried upgrading pyOpenSsl (to 0.15.1) aswell as Cryptography (not sure which one includes the other), and the problem is still not solved.

[eguiraud]

Hello, I know this issue has been closed but I just wanted to point out that looking at the diff of two strace outputs of a vim startup, one with and one without ycm enabled, you can get a better grasp of where the problem lies.

Maybe someone else will have a moment of clarity looking at this comment and will think of an easy way to solve this annoying bug forever.

I attach a diff where only the relevant lines have been left. Note the (almost) 20'000 lines spent looking for modules and libreries that are not there.

diff_woycm_wycm.txt

[puremourning]

Note the (almost) 20'000 lines spent looking for modules and libreries that are not there.

I think what you are describing is the behaviour of Python on startup. I'm afraid there isn't anything we can do to change the design of Python.

On 15 Mar 2016, at 09:51, bluehood notifications@github.com wrote:

Note the (almost) 20'000 lines spent looking for modules and libreries that are not there.

[eguiraud]

Sorry I think I am missing a piece...simply calling python opens the console instantly, and I cannot find a way to reproduce that slow python startup in any other way than opening vim+ycm.

Are you also importing many heavy modules on startup? Just trying to understand what the exact issue might be so I can tinker with things a bit