yajo
commented
3 years ago Did you find any workaround for this?
Closed jeffcasavant closed 3 years ago
yajo
commented
3 years ago Did you find any workaround for this?
abn
commented
3 years ago @jeffcasavant @Yajo I am guessing this is still happening with 1.1.0 ?
abn
commented
3 years ago Related: #2327 #2325
jeffcasavant
commented
3 years ago This seems fixed in 1.1.1 @abn . I'm curious if @Yajo is still having the issue, but I am now seeing both the .venv/src checkout and the poetry.lock updated when I do a poetry update <git dependency>.
I do have the ref marked as a branch in case that's important:
git-depend = {git = "ssh://git@github.com/<company>/<repo>.git", branch = "test-branch"}It seems the problem is reproducible when a pypi package is upgraded to a git package. Example:
Notice this line:
questionary = "^1.5.2"I'm going to test questionary. I was trying to update from git to get a patch that wasn't released yet (it is now in questionary 1.6.0, but it still can be used to test this bug):
I run:
➤ poetry --version
Poetry version 1.1.1
➤ poetry install
Creating virtualenv copier in /var/home/yajo/prodevel/copier/.venv
Installing dependencies from lock file
Package operations: 91 installs, 0 updates, 0 removals
# ... lots of stuff
➤ poetry run python -c 'import inspect; from questionary.prompts.text import text; print(inspect.signature(text))'
(message: str, default: str = '', validate: Any = None, qmark: str = '?', style: Union[prompt_toolkit.styles.style.Style, NoneType] = None, **kwargs: Any) -> questionary.question.QuestionFine. So, I'm getting version 1.5.2, without that multiline arg.
Now I updated it to the git version, pinning the commit with the fix:
➤ poetry add git+https://github.com/tmbo/questionary.git@5b8b6dec56a75970edd2bdee4dfaa5606e7c30c4
Updating dependencies
Resolving dependencies... (6.5s)
Writing lock file
No dependencies to install or updateThe important part IMHO is:
diff --git a/pyproject.toml b/pyproject.toml
index 1f8df8a..cc6ec74 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -32,7 +32,7 @@ jinja2 = "^2.11.2"
pathspec = "^0.8.0"
plumbum = "^1.6.9"
pydantic = "^1.5.1"
-questionary = "^1.5.2"
+questionary = {git = "https://github.com/tmbo/questionary.git", rev = "5b8b6dec56a75970edd2bdee4dfaa5606e7c30c4"}
regex = "^2020.6.8"
pyyaml = "^5.3.1"
pyyaml-include = "^1.2"OK, so it updated to git rev, but supposedly it said it has nothing to update... weird. Let me check:
➤ poetry run python -c 'import inspect; from questionary.prompts.text import text; print(inspect.signature(text))'
(message: str, default: str = '', validate: Any = None, qmark: str = '?', style: Union[prompt_toolkit.styles.style.Style, NoneType] = None, **kwargs: Any) -> questionary.question.QuestionWeird... I still have the version that doesn't include multiline arg. Now, trying again:
➤ rm -Rf .venv/
➤ poetry install
Creating virtualenv copier in /var/home/yajo/prodevel/copier/.venv
Installing dependencies from lock file
Package operations: 91 installs, 0 updates, 0 removals
# ... lots of stuff
• Installing questionary (1.5.2 5b8b6de)
➤ poetry run python -c 'import inspect; from questionary.prompts.text import text; print(inspect.signature(text))'
(message: str, default: str = '', validate: Any = None, qmark: str = '?', style: Union[prompt_toolkit.styles.style.Style, NoneType] = None, multiline: bool = False, instruction: Union[str, NoneType] = None, **kwargs: Any) -> questionary.question.QuestionCool! Now it does have the multiline arg.
According to this evidence, it seems to me that upgrading a dependency from a package to a git one is unreliable, unless you then delete the venv and re-install it from scratch. Not cool IMHO.
ianthetechie
commented
3 years ago Had to delete the env and start over. Hope this is fixed soon.
zyv
commented
3 years ago We have the same problem with 1.1.4, the workaround is to pip uninstall affected package before running poetry install :(
revolter
commented
3 years ago The workaround that I used was to comment out the package line in the pyproject.toml file, run poetry install, then uncomment it back and run poetry install again.
TBBle
commented
3 years ago I'm fairly sure the same issue affects path dependencies too, as I had the same problem that a path dependency had its setup.py updated with a new dependncy, plus a version bump, but poetry update would not see the changed dependency until I removed and reinstalled the package from the venv.
fbeaudo
commented
3 years ago I have the same problem (poetry not seeing updates in a package tracked from its git repo) with poetry version 1.1.4.
This is quite annoying since I that package tracked from its git repo is under constant development and thus I very often need to get its latest version. But it also depends on quite heavy dependencies. So the trick of commenting out the package in pyproject.toml, running poetry install, removing the comment, poetry install again results in uninstalling and installing again heavy dependencies. This is a quite long process and I am not sure if poetry downloads the package everytime it installs a dependency; if so that's a lot of wasted bandwidth.
It would be much better if poetry were able to detect those changes on the git repo, as it used to (I was previously running 0.0.8 and I never noticed any similar behavior).
amanda-wee
commented
3 years ago Facing this problem with a git repository on Github, also on MacOS Catalina but with poetry 1.1.4
nathanielatom
commented
3 years ago Also running into this issue, poetry install doesn't actually install the commit hash from the lock file, if the git package is already installed in the virtual env. When using pip directly the --force-reinstall option works. Does poetry support a way to pass extra CLI options through to pip? Or is it worth considering using the --force-reinstall option for all git-based dependencies?
Ubuntu 20.04 and Poetry version 1.1.4
giladreti
commented
3 years ago Same here. I dug a little into the code, and as far as I can tell the problem origins from the fact that when installing from a git repository, poetry clones the repository to a local dir and installed it as a package. Then, when listing the installed packages, poetry finds a .dist-info dir associated with this package, and concludes that it is a standart package, as seen here. Then, if no .pth files exist for the package, it just hits the continue at this line, and never reaches the conditional check at this line, thus never assigned attributes that indicate that it is a git repository.
But when solving dependencies, poetry tries to find an existing git package as can be seen here. Since the existing package hasn't been assigned with a "git" source_type, it cannot understand that the already installed package is itself a git package, and does not update it.
revolter
commented
3 years ago FIFY:
Same here. I dug a little into the code, and as far as I can tell the problem origins from the fact that when installing from a git repository, poetry clones the repository to a local dir and installed it as a package. Then, when listing the installed packages, poetry finds a
.dist-infodir associated with this package, and concludes that it is a standart package, as seen here:
Then, if no
.pthfiles exist for the package, it just hits thecontinueat this:
line, and never reaches the conditional check at this line:
thus never assigned attributes that indicate that it is a git repository. But when solving dependencies, poetry tries to find an existing git package as can be seen here:
Since the existing package hasn't been assigned with a
"git"source_type, it cannot understand that the already installed package is itself a git package, and does not update it.
ArthurTlprt
commented
3 years ago I have tried to find a fix, here is what I have tried:
During the installation of a git dependency, no .pth in is created inside the site-package of the virtual environment.
But from what I have understood, it is needed for a git dependency to have such file, specifying where the dependency source is installed.
By adding the following line into poetry, Poetry could create a .pth file
specifying the source path of the installed dependency.
Right after:
https://github.com/python-poetry/poetry/blob/adc3cde3473e85eb744f1f73d937439a681ae365/poetry/installation/executor.py#L612
I added: package.develop = True
Now Poetry is updating the git dependency. But it updates it at every poetry update.
Secondly, When poetry checks if a new version of the git dependency is available, it seems that it uses the hash of the last commit and compares it to the hash of the last commit saved into the poetry.lock.
However, in the source code, it seems that poetry compares not the right variables. As a result the variable values are always different, and it updates the dependency every time I run poetry install or poetry update.
https://github.com/python-poetry/poetry/blob/adc3cde3473e85eb744f1f73d937439a681ae365/poetry/puzzle/solver.py#L115
Because in my case, the value of locked.source_reference was master and the value pkg.source_reference was 504710cd1c300c4859b69d987ccacef6a87543d7, the hash of the last commit.
So I tried to change this line by: and locked.source_resolved_reference == pkg.source_reference.
As in my case, the value of locked.source_resolved_reference was also a commit hash.
Despite this try, poetry keeps on updating the git dependency even if no new commit is added to the dependency git repo.
I would like to find a fix on this issue, but I'm stuck for the moment. So I'd be happy to hear anybody who has a good understanding on how poetry is updating the project dependencies.
amanda-wee
commented
3 years ago https://github.com/python-poetry/poetry/issues/3740 is a duplicate of this with some work to try and understand the root of the problem.
amanda-wee
commented
3 years ago https://github.com/python-poetry/poetry/issues/3803 sounds like a related issue.
wakemaster39
commented
3 years ago I have created #3867 which resolved this issue and a couple more:
This PR supersedes my original work with #2327 to be aligned with the updated code base
galamit86
commented
3 years ago The fix is not working for me, is it just on my side? Curious to see if other people here have successfully had their git dependencies updated.
wakemaster39
commented
3 years ago @galamit86 you are correct it poetry update does not function correctly. I have updated #3867 to fix this problem so that poetry update will properly detect local git repositories
amanda-wee
commented
3 years ago I can confirm that the fix is not working for me.
nschrader
commented
3 years ago Same problem here with path-dependencies package-a = {path = "../package_a", develop = true}
Poetry 1.1.16
poetry update makes the right changes to poetry.lock and installs all changes except for package-a
Edit: same problem when changing back to a PyPI dependency. poetry update even logs
Updating package-a (1.0.1 /tmp/tmp9hldnj_b/package-a -> 1.0.1)but doesn't install anything.
thehappydinoa
commented
3 years ago Hi, I am still dealing with the same issue. Is there anything we can do to have this escalated/reopened? Maybe @sdispater or @abn ?
aptlin
commented
2 years ago Any updates?
Ivorforce
commented
2 years ago Here's the smallest workaround I've found helps:
python -c 'import importlib; import sys; print("\n".join([importlib.import_module(m).__path__[0] for m in sys.argv[1:]]))' package_1 package_2 [...]
rm -rf <package folder>* # for each of the outputs
poetry installEffectively you're removing the local caches and poetry installs them again. This is a lot faster than asking it to re-build the whole cache twice.
inspiralpatterns
commented
2 years ago I am still experiencing the same problem with Poetry version 1.1.7 on Mac OS Big Sur 11.5.2.
I tried to switch between using branch and rev when defining the dependency in pyproject.toml, but it didn't work either way.
My workaround is to delete the venv and run poetry install again.
sify21
commented
2 years ago I am still experiencing the same problem with
Poetry version 1.1.7on Mac OS Big Sur 11.5.2. I tried to switch between usingbranchandrevwhen defining the dependency inpyproject.toml, but it didn't work either way. My workaround is to delete thevenvand runpoetry installagain.
me too, maybe we should reopen this. I'm using fedora 34
Ivorforce
commented
2 years ago TL; DR: For a fix, run poetry self update --preview
There is a new open issue for this topic. Please check out #3958 instead. After this time I don't think this issue will be re-opened.
nikiladonya
commented
2 years ago I have similar issue
if you change branch of poetry git dependency and do poetry update and poetry install, nothing changes
Congee
commented
2 years ago This issue at least persists for some users. Can it be reopened?
vale981
commented
2 years ago This issue at least persists for some users. Can it be reopened?
justinbarak
commented
2 years ago Issue still persists for me as well.
slyons
commented
2 years ago I'm also still encountering this. Having a force flag on update/install would be useful in this situation.
uselessvevo
commented
2 years ago Will this issue ever fixed?
vale981
commented
2 years ago Will this issue ever fixed?
Andrew-Wichmann
commented
1 year ago I was having this issue until I tried removing the entire virtualenv + poetry.lock file, adding my git dependency with the develop=true flag, then trying again.
# pyproject.toml
[tool.poetry.dependencies]
...
my_library = { git = "git@gitlab.com:my_library.git", branch = "test", develop = true}
miohtama
commented
11 months ago As a workaround, if you do not care about any proper fix and want to have just the files from the latest Git branch installed, you can do pip install within the Poetry virtual environment:
poetry shell
pip install -e "git+https://github.com/myorg/myrepo.git@master#egg=mypackage[extra1,extra2]"
djaney
commented
6 months ago I had similar issue where it seems to pull the wrong package meta - pulling outdated. Running poetry config experimental.system-git-client true worked for me. It seems the issue was the git client used by poetry. Had to use the system default.
github-actions[bot]
commented
4 months ago This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
-vvvoption).Issue
I have a git dependency that is defined with a branch instead of a specific commit or revision. As a result, the commit that the reference points to can change.
poetry updatesuccessfully updates thepoetry.lockfile with the correct latest commit, but the library in the virtual environment remains checked-out at the commit that was latest whenpoetry installwas last run.Replication
poetry installpoetry shell,cd $VIRTUAL_ENV/src/<git dependency name> && git log | head -1)poetry.lock(cat poetry.lock | grep -i <git dependency name> -B 2 | grep -i referenceworked for me, but obviously YMMV; you're looking for thereferencefield in thepackage.sourceblock with theurlequal to the git repository you installed from)poetry update <git dependency name>poetry.lockhas changedsrchas not changedI expected that since
poetry.lockwas updated that my virtualenv was updated as well.