search

koxudaxi / poetry-pycharm-plugin

A PyCharm plugin for poetry
https://koxudaxi.github.io/poetry-pycharm-plugin/
Apache License 2.0
180 stars 14 forks source link

Virtualenv gets deleted when I enable Poetry plugin #160

Open intgr opened 3 years ago

intgr commented 3 years ago

Describe the bug

Whenever I start IDEA with the Poetry plugin enabled, it starts indexing the virtualenv and within seconds the whole $VIRTUAL_ENV directory gets completely removed. No new virtualenv is created in its place. This only occurs in one project though.

  1. IDEA complains "Invalid Python interpreter selected for the module".
  2. I restore the virtualenv from backup
  3. When I click on the "Configure Python Interpreter" on IDEA's notification, immediately it starts deleting files, which takes a few seconds until completely removed.
  4. Go to 1

This started occurring in within the last week, I performed a 'brew update' and update to IDEA plugins. Until then I had been using the Poetry plugin without issues.

Disabling the Poetry plugin solves this issue.

I am not sure how to debug this further, nothing relevant appears in idea.log file. Is it possible to log all commands that the Poetry plugin invokes?

Environments (please complete the following information):

intgr commented 3 years ago

I noticed that Poetry has some built-in "broken virtualenv detection", which I can trigger manually by deleting bin/python. But I don't know why the Poetry plugin would be triggering this behavior.

% rm $VIRTUAL_ENV/bin/python
% poetry shell
The virtual environment found in /.../Library/Caches/pypoetry/virtualenvs/idea-bugs-H1yE5bTC-py3.9 seems to be broken.
Recreating virtualenv idea-bugs-H1yE5bTC-py3.9 in /.../Library/Caches/pypoetry/virtualenvs/idea-bugs-H1yE5bTC-py3.9
Virtual environment already activated: /.../Library/Caches/pypoetry/virtualenvs/idea-bugs-H1yE5bTC-py3.9

Since I also have poetry installed within the virtualenv, it may fail to re-create itself after deleting.

intgr commented 3 years ago

I also tried deleting and re-creating the IDEA Python plugin interpreter configuration, but the problem persists.

koxudaxi commented 3 years ago

@intgr I'm sorry for the late reply.

I am not sure how to debug this further, nothing relevant appears in idea.log file. Is it possible to log all commands that the Poetry plugin invokes?

No. We can't save logs of the commands. We may add a feature to get logs.

I tested it on my environment close to your environment. But, I couldn't get the same error.

My environment IDE: IntelliJ IDEA 2020.3.1 (Ultimate Edition) OS: macOS 10.15.7 Poetry Version 1.1.4 Plugin version 1.0.2

Since I also have poetry installed within the virtualenv, it may fail to re-create itself after deleting.

I don't recommend to install poetry within the virtualenv. I think pipx is good tools to install poetry. https://python-poetry.org/docs/#installing-with-pipx

intgr commented 3 years ago

I don't recommend to install poetry within the virtualenv.

Yeah I would very much like to get rid of it, but it's currently a necessary workaround for us until this PR goes anywhere: https://github.com/python-poetry/poetry/pull/3349

joelb123 commented 3 years ago

I can confirm this is an issue for me. It makes the plugin totally unusable. I install a virtualenv, and within seconds it gets auto-deleted: ``OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

I believe it's the auto-dispose of the python interpreter that's doing the crime here.

Environment: Poetry Version 1.1.4 installed in external environment Plugin Version 1.0.2 I used 1.0.1 extensively and it did not do this under Pycharm 3.2.0 Pycharm 3.2.2 Gentoo Linux kernel 5.10.6 OpenJDK 64-bit server built by JetBrains

joelb123 commented 3 years ago

I reinstalled plugin version 1.0.1 and verified that the auto-dispose of the python interpreter does not happen with the plugin enabled.

wgordon17 commented 3 years ago

This is biting me now too!

PyCharm 2020.3.4 (Professional Edition)
Build #PY-203.7717.65, built on March 16, 2021
Runtime version: 11.0.10+8-b1145.96 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 10.16
GC: ParNew, ConcurrentMarkSweep
Memory: 4416M
Cores: 12
Non-Bundled Plugins: org.intellij.plugins.postcss, com.khmelyuk.multirun, com.paperetto.dash, fr.mary.olivier.aw-watcher, ideanginx9, net.seesharpsoft.intellij.plugins.csv, mobi.hsz.idea.gitignore, org.toml.lang, NodeJS, com.intellij.plugins.html.instantEditing, me.kapien.vue-router-helper, org.jetbrains.plugins.vue, com.jetbrains.intellij.datalore, ru.adelf.idea.dotenv, ru.meanmail.plugin.requirements
$ poetry debug info

Poetry
Version: 1.1.5
Python:  3.9.1

Virtualenv
Python:         3.9.1
Implementation: CPython
Path:           NA

System
Platform: darwin
OS:       posix
Python:   /Users/wgordon/.pyenv/versions/3.9.1

Any advice or recommendation would be greatly appreciated!

wgordon17 commented 3 years ago

I seem to be able to hit this consistently by removing the python interpreter from PyCharm (I've confirmed that removing the interpreter from PyCharm doesn't touch the directory on disk), and then re-adding it. The deletion appears to occur once I hit "OK" and during the "Loading Packages L..." phase that I see along the bottom of the PyCharm window.

image

image

Going from 1.0.6, all the way down to 1.0.0, I was still able to consistently reproduce this issue. I've also confirmed that this seems to just be affected by the Poetry plugin. If I add the same interpreter as a "Virtualenv Environment", I don't see the same behavior.

koxudaxi commented 3 years ago

@wgordon17 Thank you for show me the detail. I can't reproduce the problem 🤔

Going from 1.0.6, all the way down to 1.0.0, I was still able to consistently reproduce this issue.

I just checked the histories of changes. But, you said the changes are not related :(

wgordon17 commented 3 years ago

Weird! Is there any way to enable debugging or audit logging in PyCharm? I'm happy to provide any additional information if necessary!

koxudaxi commented 3 years ago

Weird! Is there any way to enable debugging or audit logging in PyCharm? I'm happy to provide any additional information if necessary!

Thank you.

The simple answer is nothing :( Of course, you can run a plugin from a source with a debugger in IDEA. But, this way is not easy.

Would you tell me how do you reproduce it?

My environment Poetry: 1.1.5 PyCharm: 2020.3.4 Python: 3.9.2 macOS: 10.15.7

joelb123 commented 3 years ago

This might be a complete red herring, but the other problem that was driving me crazy was JCEF crashing with the ReStructuredText plugin enabled due to problems accessing the GPU. I solved that by using the system java rather than bundled; I believe that may have solved the this problem, but I'm not in a situation to test this atm.

koxudaxi commented 3 years ago

Thank you for the comment. The viewpoint may be good. Because @wgordon17 said, the problem happened in 1.0.0. The version has the same similar implementation as Pipenv integration in PyCharm. It means it does not include the additional features for only poetry.

koxudaxi commented 3 years ago

@sproshev Could you give us some advice on the problem?

In the latest version, I had added some features to checking poetry executable. The feature is sync function that blocks PyCharm. I was afraid the feature may cause the problem. But, The problem happened older version too which doesn't have the feature.

wgordon17 commented 3 years ago

I hope this helps https://www.youtube.com/watch?v=Xt5WEfaYzrA

I'm really not sure where the issue is ¯\_(ツ)_/¯ At first, I was totally clueless, so then I used opensnoop to track it down. I could clearly see python3.9 creating things (presumably poetry install), and then I would see a single -1 from pycharm, which I can only assume means "delete".

Once I realized it was PyCharm I updated to 2020.3.4. And finally, at my whits end, I stumbled across this issue. I honestly don't recall this being an issue in the past, but then again, how often does someone switch out there Project Interpreter. It definitely seems to be an issue with older versions of your Poetry plugin, but whether it's an interaction with newer versions of PyCharm...who the heck knows ¯\_(ツ)_/¯

I'm at least back to productive again using the "Virtualenv Environment" interpreter, but I definitely miss your plugin's "magic" ;)

koxudaxi commented 3 years ago

@wgordon17 Thank you for uploading the video. Today, I tried to reproduce your problem on my machine. But, I can't see it. Also, I have never seen the error. But, I don't give up to fix the problem.

sproshev commented 3 years ago

@koxudaxi we have a similar issue with pipenv https://youtrack.jetbrains.com/issue/PY-36526 but I'm not sure if it is actually related.

koxudaxi commented 3 years ago

@sproshev Thank you for the link. I will check the issue.

svenski commented 3 years ago

The behaviour described in here started happening for me today, out of the blue. A poetry install from the command line resets the environment but as soon as I open pycharm, it get's deleted. A workaround that has worked so far is to re-create the poetry env from within pycharm. It survived a re-start.

Peotry Pycharm Plugin version: 1.1.4

Poetry version 1.1.6

Pycharm Version PyCharm 2021.1.2 (Professional Edition) Build #PY-211.7442.45, built on June 1, 2021

Runtime version: 11.0.11+9-b1341.57 amd64 VM: Dynamic Code Evolution 64-Bit Server VM by JetBrains s.r.o. Linux 5.8.0-59-generic GC: G1 Young Generation, G1 Old Generation Memory: 2048M Cores: 4 Non-Bundled Plugins: IdeaVIM (0.68), Key Promoter X (2021.1.1), net.seesharpsoft.intellij.plugins.csv (2.17.0), com.koxudaxi.poetry (1.1.4), com.koxudaxi.pydantic (0.3.3) Current Desktop: Regolith:GNOME-Flashback:GNOME

phillipuniverse commented 2 years ago

Also ran into this today. Thanks to @svenski this workaround worked!

A workaround that has worked so far is to re-create the poetry env from within pycharm. It survived a re-start.

Allowing Pycharm to create the Poetry venv allowed me to utilize poetry from both Pycharm and the command line (e.g. poetry env info).

PyCharm 2021.2.2 (Professional Edition)
Build #PY-212.5284.44, built on September 14, 2021
Licensed to Shipwell / Phillip Verheyden
Subscription is active until February 20, 2022.
Runtime version: 11.0.12+7-b1504.28 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 11.5.2
GC: ParNew, ConcurrentMarkSweep
Memory: 4012M
Cores: 12
Non-Bundled Plugins: org.sonarlint.idea (5.3.0.36775), com.github.hinaser.gfm-advanced (0.0.8), com.github.redfoos.logstash-intellij-plugin (0.65-EAP), org.intellij.plugins.hcl (0.7.14), name.kropp.intellij.makefile (212.4746.52), ski.chrzanow.foldableprojectview (1.1.1), org.asciidoctor.intellij.asciidoc (0.33.19), mobi.hsz.idea.gitignore (4.2.0), com.intellij.swagger (212.5080.16), com.dmarcotte.handlebars (212.4746.57), org.toml.lang (0.2.155.4114-212), org.rust.lang (0.4.155.4114-212), com.koxudaxi.poetry (1.1.5-212), com.jetbrains.intellij.datalore (0.1.13-212), ru.meanmail.plugin.requirements (2021.4.1-212), com.leinardi.pycharm.pylint (0.12.2), ru.adelf.idea.dotenv (2021.3.0.212), com.koxudaxi.pydantic (0.3.8), com.dropbox.plugins.mypy_plugin (0.3.2), aws.toolkit (1.32-212)