Support for python VCS package version update triggers

[MrKevinWeiss]

Is there an existing issue for this?

• [X] I have searched the existing issues

Feature description

Hey everyone :wave:

Dependabot seems really great and a nice way to reduce the overhead of using many separate repos. I have been struggling to find some information on how to resolve python packages that do not exist on pypi.

For example, if I have a requirement.txt or Pipfile or something that contains a vcs link to a python package, how can I check to see if something has been updated?

I am aware of that using VCS with pip has limitations, for example, I don't think >= would work...

For example if I have a requirements.txt file:

git+https://github.com/me/pkg1#egg=pkg1==0.1.0

and I release a 0.2.0 in the "default" branch, should that trigger a version update then? with pip install I get a

  WARNING: Requested pkg1==0.1.0 from git+https://github.com/MrKevinWeiss/pkg1#egg=pkg1==0.1.0 (from -r requirements.txt (line 2)), but installing version 0.2.0

message so it is aware that there is a version mismatch...

The alternative would be using tags/commit hashes with the editable version. In that case I shouldn't it just check if the tag or so has been changed in the tree? Is there any option to allow that?

Is there a preferred method for managing dependencies with (git) repo based python packages?

I did notice that the Pipfile shows the pkg1 on the dependency graph, while the requirements.txt does not. Most of my trials has been with requirements, should I repeat with Pipfile or another package manager file?

Maybe even just pointing to some examples where this is used as #3494 seems like a step beyond (ie, dealing with private repos).

Lots of questions, I know, thanks for looking!

[pavera]

Unfortunately pip itself has been changed and no longer parses versions from VCS requirements in a requirements.txt file. This prevents us from being able to accurately compare versions to determine if an update is needed.

I would recommend investigating pipenv/Pipfile or poetry though I don't know if those will work either, only that Dependabot does not currently support updating VCS dependencies defined in a requirements.txt file. If you find a working solution or learn anything specific to pipenv or poetry in this investigation we'd love to hear about it here.

[MrKevinWeiss]

If you find a working solution or learn anything specific to pipenv or poetry in this investigation we'd love to hear about it here.

Thanks for the clarification, I will continue investigation and update if I find something useful!

[MrKevinWeiss]

Ok, I tried with Pipfile (pipenv) and with the pyproject.toml (poetry), with branches, versions, editable modes, tags, and commit hashes. Nothing seems to trigger updates but I do see the Pipfile and pyproject.toml VCS links at least. I tried also with a requirements.in file but it didn't seem to do much (and that would also be my least favorite way of managing things).

I have yet to try with URL artifacts but I would assume that it probably will not work.

This has not been exhaustive but so far I would say that VCS based python version checks are not supported... If anyone has had some success with trigger a version bump PR based off of a VCS (preferably git) python package dependency please let me know what the conditions were. Otherwise I would humbly request that feature!

[MrKevinWeiss]

I also tried with the insecure-external-code-execution allowed and no luck... I guess I can use the gitsubmodule as a workaround (though it is not as clean as with pip)...