Closed ramonski closed 4 years ago
If it actually freezes completion, I'd be happy to know what causes it. Are there too many modules? Is the freeze permanent? Does the freeze reoccur when completing again?
I'd like to fix the bug instead of creating some workaround options that almost nobody will ever use, buildout itself is pretty out of fashion and an ignore option... That would probably just be one user: you ;-).
Hi @davidhalter, thanks for your fast reply.
There are actually a lot of modules in the project, but without the bin
directory containing the Python scripts in the root directory, everything works really fast.
It seems like the freeze is not permanent, but runs in the 30s timeout of anaconda-mode
:
https://github.com/proofit404/anaconda-mode/blob/master/anaconda-mode.el#L165
The freeze affects all completions and is ongoing, probably because these scripts manipulate the sys.path
when they are loaded at runtime.
I've seen that the duplicates are removed here: https://github.com/davidhalter/jedi/blob/master/jedi/api/project.py#L124
but this is just done once and probably every Python module which does the sys.modules
manipulation as above will cause trouble depending on the amount of paths they insert.
In my case each script has about 300 path insertions and inside the bin
folder there are about 15 such scripts.
Anyhow, I would indeed completely remove the buildout integration, because I do not see any value in having code completion for auto-generated Python scripts in my project folder – actually I think this project integration (the Django one as well) shouldn't be in the domain of Jedi, but from the user.
But when these sys.path
hacks cause such trouble, it would make probably sense to not allow any sys.path
modifications at all from loaded modules?
Just guessing, I didn't go into much detail and I was a bit lost in how to setup Jedi from the command line and make it call that buildout integration code to put a PDB there...
But when these sys.path hacks cause such trouble, it would make probably sense to not allow any sys.path modifications at all from loaded modules?
They don't cause much trouble, they are actually a help for most people. Even the buildout integration. If you're adding 300 paths, you are probably the exception and not the rule. I can see that this probably doesn't perform very well. Jedi will open all eggs and unzip them?! I would need a better understanding of what happens to limit this.
Can you use a straight Jedi call and reproduce it this way? Then I would need a python -m cProfile script.py
output.
Yes of course I can try to do so, but I wasn't able to write the code in such that this method was called: https://github.com/davidhalter/jedi/blob/master/jedi/api/project.py#L109
In my Emacs I did a very simple test and added a foo.py
module with this contents:
import json
json.lo|
Inside the project folder containing the buildout.cfg
it ran into a timeout, while one level above this folder it worked fast. Removing the bin
folder inside the project folder resolved the issue and the code completion was almost immediately.
Removing the scripts one by one of the bin
folder made completion go faster, but even 1 Python script like the one I outlined above reduced the performance immense.
What I tried was to do it like described here: https://jedi.readthedocs.io/en/latest/docs/api.html#module-jedi similar to what I did in Emacs:
Python 2.7.16 |Anaconda, Inc.| (default, Mar 14 2019, 16:24:02)
Type "copyright", "credits" or "license" for more information.
IPython 5.8.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import jedi
In [2]: jedi.__version__
Out[2]: '0.13.3'
In [3]: source = """
...: import json
...: json.lo"""
In [4]: script = jedi.Script(source, 3, len("json.lo"), "foo.py")
In [5]: script
Out[5]: <Script: 'foo.py' <SameEnvironment: 2.7.16 in /Users/rbartl/miniconda2/envs/senaite>>
In [6]: script.completions()
Out[6]: [<Completion: load>, <Completion: loads>]
In [7]: ls
CHANGES.rst CONTRIBUTORS.rst README.rst bika/ buildout.cfg parts/ setup.py templates/
CODE_OF_CONDUCT.md LICENSE RELEASE_NOTES.rst bin/ develop-eggs/ senaite.core.egg-info/ src/ travis.cfg
CONTRIBUTING.md MANIFEST.in __init__.py bootstrap.py eggs/ setup.cfg static/ var/
However, this works pretty fast but does not seem to execute the same code like here: https://github.com/proofit404/anaconda-mode/blob/master/anaconda-mode.el#L167
I'm using a Miniconda environment, so no Virtualenv involved here.
Yes of course I can try to do so, but I wasn't able to write the code in such that this method was called: https://github.com/davidhalter/jedi/blob/master/jedi/api/project.py#L109
Please try again, Make sure to be in the buildout folder and add a path to Script
. I cannot help you without such output.
Ok, this are the contents of the script foo.py
I dropped into the project folder:
import jedi
print jedi.__version__
source = """
import json
json.lo
"""
script = jedi.Script(source, 3, len("json.lo"), "foo.py")
print script
print script.completions()
The bin
folder in this Project directory contains these files:
$ ls -lah bin
total 576K
drwxr-xr-x 49 rbartl staff 1,6K Mai 20 08:43 .
drwxr-xr-x 27 rbartl staff 864 Mai 21 14:23 ..
-rwxr-xr-x 1 rbartl staff 2,8K Mai 19 08:53 backup
-rwxr-xr-x 1 rbartl staff 313 Mai 19 08:52 buildout
-rwxr-xr-x 1 rbartl staff 1,3K Mai 20 08:43 check-manifest
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-chameleon-lint
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-check-manifest
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-clean-lines
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-csslint
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-dependencychecker
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-find-untranslated
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-flake8
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-importchecker
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-jscs
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-jshint
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-scsslint
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-xmllint
-rwxr-xr-x 1 rbartl staff 3,4K Mai 20 08:43 code-analysis-zptlint
-rwxr-xr-x 1 rbartl staff 22K Mai 20 08:43 createfontdatachunk.py
-rwxr-xr-x 1 rbartl staff 391 Mai 19 08:53 develop
-rwxr-xr-x 1 rbartl staff 23K Mai 20 08:43 enhancer.py
-rwxr-xr-x 1 rbartl staff 24K Mai 20 08:43 explode.py
-rwxr-xr-x 1 rbartl staff 1,4K Mai 20 08:43 flake8
-rwxr-xr-x 1 rbartl staff 22K Mai 20 08:43 gifmaker.py
-rwxr-xr-x 1 rbartl staff 677 Mai 20 08:43 i18ndude
-rwxr-xr-x 1 rbartl staff 22K Mai 20 08:43 instance
-rwxr-xr-x 1 rbartl staff 23K Mai 20 08:43 painter.py
-rwxr-xr-x 1 rbartl staff 24K Mai 20 08:43 pilconvert.py
-rwxr-xr-x 1 rbartl staff 36K Mai 20 08:43 pildriver.py
-rwxr-xr-x 1 rbartl staff 24K Mai 20 08:43 pilfile.py
-rwxr-xr-x 1 rbartl staff 22K Mai 20 08:43 pilfont.py
-rwxr-xr-x 1 rbartl staff 24K Mai 20 08:43 pilprint.py
-rwxr-xr-x 1 rbartl staff 23K Mai 20 08:43 player.py
-rwxr-xr-x 1 rbartl staff 722 Mai 19 21:27 repozo
-rwxr-xr-x 1 rbartl staff 2,8K Mai 19 08:53 restore
-rwxr-xr-x 1 rbartl staff 21K Mai 20 08:43 robot
-rwxr-xr-x 1 rbartl staff 21K Mai 20 08:43 robot-server
-rwxr-xr-x 1 rbartl staff 2,8K Mai 19 08:53 snapshotbackup
-rwxr-xr-x 1 rbartl staff 2,8K Mai 19 08:53 snapshotrestore
-rwxr-xr-x 1 rbartl staff 258 Nov 7 2018 sync_dbs.sh
-rwxr-xr-x 1 rbartl staff 20K Mai 20 08:43 test
-rwxr-xr-x 1 rbartl staff 23K Mai 20 08:43 thresholder.py
-rwxr-xr-x 1 rbartl staff 283 Mai 20 08:43 update_sources
-rwxr-xr-x 1 rbartl staff 2,9K Mai 20 08:43 update_translations
-rwxr-xr-x 1 rbartl staff 22K Mai 20 08:43 viewer.py
-rwxr-xr-x 1 rbartl staff 2,5K Mai 20 08:43 write_code_headers
-rwxr-xr-x 1 rbartl staff 21K Mai 20 08:43 zope_health_watcher
-rwxr-xr-x 1 rbartl staff 22K Mai 20 08:43 zopepy
This is the output of the call python -m cProfile foo.py
with the bin
folder (3.599 seconds):
Hi all,
I encountered some strange behaviour in my
buildout.cfg
based Plone projects, which brought me here.I'm using Spacemacs as my editor with anaconda-mode and company-anaconda for code completion.
Everything works fine outside of my zc.buildout based projects and I get code completion really fast for my Python code.
However, inside these project folders the process seem to hang and now I found out, that it is because of the scripts inside the
bin
folder, which get automatically added in thesys.path
by this code:https://github.com/davidhalter/jedi/blob/master/jedi/evaluate/sys_path.py#L179
Most of these scripts modify the
sys.path
as well and look like this:Having all these scripts inside, seem to block the process and no further completion is possible.
Renaming the
bin
folder to_bin
solves all issues and I get the normal behaviour as outside of the project folder.I hadn't the chances yet to look into detail yet, but it doesn't seem that this can be turned off by a configuration option, so my question would be, if someone else encountered this issue as well and if it would make sense to make this behaviour optional. Maybe a
.ignore
marker inside a folder to skip traversing?Thanks