Closed DonJayamanne closed 6 years ago
xobs
commented
6 years ago I'm experiencing this issue as well.
Relatedly, my vscode is currently in a state where it is not analyzing the workspace, vscode is idle, but Microsoft.Python.LanguageServer.exe is using 22% CPU. So while Analyzing workspace, ${n} items remaining... is one indicator of the issue, it's not the only indicator, and sometimes the language server can spin on its own.
DonJayamanne
commented
6 years ago Happened again today (on a Mac):
derekwallace
commented
6 years ago I experience this issue aswell.
ijstokes
commented
6 years ago Here too. Not sure if it is helpful, but I'm also having this pop up in the OUTPUT tab:
[Error - 07:34:56] System.NullReferenceException: Object reference not set to an instance of an object.
at System.Object.GetType()
at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.EvaluateWorker(Node node)
at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(ClassDefinition node)
at Microsoft.PythonTools.Parsing.Ast.ClassDefinition.Walk(PythonWalker walker)
at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(SuiteStatement node)
at Microsoft.PythonTools.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker)
at Microsoft.PythonTools.Analysis.Analyzer.DDG.WalkBody(Node node, AnalysisUnit unit)
at Microsoft.PythonTools.Analysis.ClassAnalysisUnit.AnalyzeWorker(DDG ddg, CancellationToken cancel)
at Microsoft.PythonTools.Analysis.Analyzer.DDG.Analyze(Deque`1 queue, CancellationToken cancel, Action`1 reportQueueSize, Int32 reportQueueInterval)
at Microsoft.PythonTools.Analysis.PythonAnalyzer.AnalyzeQueuedEntries(CancellationToken cancel)
at Microsoft.PythonTools.Intellisense.AnalysisQueue.GroupAnalysis.Analyze(CancellationToken cancel)
at Microsoft.PythonTools.Intellisense.AnalysisQueue.Worker(Object threadStarted)(and @DonJayamanne has Microsoft taken over the vscode-python repo? IOW, is this the main repo to track and report issues to?)
ijstokes
commented
6 years ago Here's a screenshot. I've disabled pretty much every VSCode extension. Reported also here: https://github.com/Microsoft/vscode/issues/55815
ijstokes
commented
6 years ago OK, getting to be chronic now. I've turned off most extensions, now down to just Python and Vim, and I'm getting this every 1-2 hours. System stats below.
Impact on me: vim bindings stop working.
VS Code version: Code 1.25.1 (1dfc5e557209371715f655691b1235b6b26a06be, 2018-07-11T15:33:29.235Z) OS version: Darwin x64 17.7.0
DonJayamanne
commented
6 years ago @ijstokes
IOW, is this the main repo to track and report issues to?
Yes
Please could you create a separate issue for the problem you are facing. There are no errors displayed/reported by the language server in the original issue. Where as the issue you have reported, has errors being reported by the language server, so please create a separate issue. Thanks
MikhailArkhipov
commented
6 years ago @zooba may be interested in https://github.com/Microsoft/vscode/issues/55815 as well.
zooba
commented
6 years ago The frequent "reloading modules" looks like either the file watcher is detecting changes or the language server is crashing and restarting frequently.
The stack trace above probably means we need to use EvaluateMaybeNull somewhere while walking class statements. I guess we ended up with a null base class or something?
patrys
commented
6 years ago I've attached to a frozen language server process with VS. Its memory profile was a flat line at 6 GB (did it run out of memory and desperately try to malloc in a loop?) and its CPU profile was oscillating around maxing out one core. Unfortunately by that point the process no longer responded to debugger break requests so I could not see where it was stuck and once restarted I could no longer reproduce.
keattang
commented
6 years ago Is there a way to disable the language server? I have the same issue and it maxes out my CPU and grinds my computer to a halt every time I open VSCode making it unusable :( I haven't been able to find any instructions on how...
d3r3kk
commented
6 years ago @keattang, you can disable the new Python Language Server by opening settings in VSCode (Ctrl+,) and set "python.jediEnabled": true. Then reload the window and/or restart VSCode.
zooba
commented
6 years ago @patrys @keattang If you are able to share the code and configuration that's getting you into this state (email links/details to ptvshelp@microsoft.com) then that can help us reproduce your issue and figure out the problem.
Unfortunately, there are still a few edge cases where certain code constructs can lead to explosive growth, but without the actual code that causes the problem there's not much we can do to detect them :(
patrys
commented
6 years ago @zooba If I was able to reproduce this easily, I'd debug this further locally; unfortunately it's not an easy thing to reproduce.
zooba
commented
6 years ago @patrys Understood. We do have a few tools that can help us reproduce and diagnose issues, though they're not really easy to use without understanding how the internal analysis model works (and that in itself is nearly a six-week course ;) One of the biggest blockers to contributions in this area is simply that this is pretty complex code)
AssenD
commented
6 years ago Almost same here. Windows 7, Anaconda, Python 3.6.5, 16 GB RAM Language Server grows up on memory constantly until memory just ends. Have to disable it.
keattang
commented
6 years ago Thanks @d3r3kk. Interestingly, I didn't actually have jedi turned off. I had recently enabled the Visual Studio IntelliCode Preview though so I wonder if that turned the language server on in the background? After turning jedi off and then back on again I am now able to use VSCode again.
@zooba Unfortunately it was a fairly large project that vscode was struggling on so I couldn't tell you which part of it caused the issue
zooba
commented
6 years ago @keattang Honestly (and I mean absolutely no disrespect by this), you wouldn't be able to tell which part was the problem anyway. The places where the language server can get hung up on certain code constructs is highly unpredictable, which is why the best way for us to find the issues is by starting with the full codebase, bisecting to find the minimum set of files to reproduce it, and then live debugging it to find the (typically) comparisons that are not properly commutative or transitive.
And unfortunately, when we can't get enough of the codebase to reproduce the issue (as is often the case), there's basically nothing we can do. All of the issues we can think of have had unit tests for years, but you can never be imaginative enough with this kind of problem :)
MikhailArkhipov
commented
6 years ago @keattang - one possibility is to set python.analysis.traceLogging to true - this will produce a lot of traffic in the output - then copy the log output and attach it here as a text file. May give us an idea where/why it is looping.
keattang
commented
6 years ago @zooba fair enough, I'd share the codebase but its a private repo owned by my employer.
@MikhailArkhipov I'll see what I can find with that
DonJayamanne
commented
6 years ago Still experiencing this issue with the latest version.
Its been analyzing for over 24 hours.
Will try once again with python.analysis.traceLogging enabled.
wskdsgcf
commented
6 years ago Experiencing this issue as well.
TortelliniLeap
commented
6 years ago Also experiencing this issue. It's been stuck analyzing 2 items for several hours. Completely restarting VS Code does not resolve it.
patrys
commented
6 years ago I'm experiencing this currently on my Mac with an insiders build of Code and Python 2018.8.0. Unfortunately I don't have the tools to debug the language server but I've managed to capture the trace output (it's an open source project so I'm fine with posting the directory structure):
It looks like it's done without errors yet the status bar is showing
Analyzing workspace, 553 items remaining...
sneridagh
commented
6 years ago Same here, CPU spinning to 100% "Analyzing workspace". I have a large one, but who that works at a professional level doesn't?
What did it "analyze"?
allenyllee
commented
6 years ago I've encountered the same problem in my ubuntu 18.04 64bit with vscode 1.27.1, vscode-python 2018.8.0 and python 3.6.5
here is the trace output log:
and the result of top command:
Although CPU has idled while this small project, the memory consumption seems too high.
the project is https://github.com/fxsjy/jieba
MikhailArkhipov
commented
6 years ago Remaining items in the status bar may be UI issue if CPU load is low. If LS was restarted b/c of exception (most probable cause is https://github.com/Microsoft/python-language-server/issues/72) then UI just shows string from the previous instance.
MikhailArkhipov
commented
6 years ago 
richardwb
commented
6 years ago One thing I noticed here is some symlink folder structures can cause really deep parsing queues to form.
Recursive/looping symlinks can end up propagating pretty deeply and if there's actual content anywhere along the way it will take a long time. For example, given a simple folder structure like this:
/code
/test
/some
test.py
/folder
/symlink -> ../../some
__init__.py (empty)
lib.py (empty)And test.py is simply "import folder.lib" so we start traversing the tree, I see the following trace output:
Initializing for /usr/local/opt/python/bin/python2.7
Loading files from /Users/username/code/test/some
Parsing document file:///Users/username/code/test/some/test.py
Parsing document file:///Users/username/code/test/some/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/test.py
Parse complete for file:///Users/username/code/test/some/test.py at version -1
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Analysis queued for file:///Users/username/code/test/some/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
<more>
Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py at version -1
Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.pyI managed to work around this by just adding the suspect symlinks to my VSCode excludes.
There's also some other odd behavior with dead symlinks, but it doesn't seem to act as poorly.
sneridagh
commented
6 years ago Symlink could cause it... I have tons of them (I use zc.buildout, with an extension (omelette) that makes all the eggs used by the project available in the same tree, using symlinks).
I'm not the only one using it... in Zope/Plone world is quite usual that devs use these tools.
It would be great if we can fix it somehow.
samuelpcabral
commented
6 years ago Same thing here! Windows 10, with Python 2.7
and it looks like it's consuming far more memory than it should.
mungojam
commented
6 years ago I wondered if it is processing my ./.venv pipenv package folder or some other temporary sub-folder. The number of items it said it was processing seemed much larger than my code base.
sneridagh
commented
6 years ago True, I didn't take into account that virtualenv also uses symlinks... don't know about pipenv, but I guess it also does it. So, there's a major problem then that might affect the vast majority of people.
xobs
commented
6 years ago It's happening on Windows as well, which doesn't use symlinks for virtualenv (that I know of). Also, my project doesn't use virtualenv.
However, it does use environment variables and mess with PYTHONPATH, which may be related to the cause.
MikhailArkhipov
commented
6 years ago
Environment data
Actual behavior
Progress message does not go away
Expected behavior
Progress in status bar must go away.
Steps to reproduce:
Logs
Output for
Pythonin theOutputpanel (View→Output, change the drop-down the upper-right of theOutputpanel toPython)Output from
Consoleunder theDeveloper Toolspanel (toggle Developer Tools on underHelp)