Full transitive dependency information

[SergeBakharev]

Is your feature request related to a problem? Please describe. For the purposes of Software Composition Analysis the ability to export a full transitive dependencies list (including deps of 3rd party reqs) is required, and in a format such as PEP508 (requirements.txt). Currently ./pants dependencies --transitive provides only the direct 3rd party dependencies in a pants target format, however the full information is present in the lock file.

Describe the solution you'd like I believe this functionality can be split into two enhancements to the dependencies goal:

• an additional flag to show reqs transitive reqs
• an additional flag to format this output in in a requirements.txt syntax. I see adjacent work is being done in #17721 with groundwork completed in https://github.com/pantsbuild/pants/commit/20b37107903053d65cf8845fd01878c7045ab99e

I would be interested in helping contribute this functionality.

Describe alternatives you've considered My current alternative is synthesize such a requirements.text file manually by pex lock exporting on the complete lock file, then removing unrelated items.

Though for components with pex targets running the built pex via PEX_INTERPRETER=1 and then running this snippet is quicker:

import pkg_resources
installed_packages = pkg_resources.working_set
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
     for i in installed_packages])
print(installed_packages_list)

[benjyw]

Thanks for opening this issue @SergeBakharev !

One thing to think about is this:

Today, dependencies only displays targets, and it displays those targets' addresses. Transitive requirements from a lockfile are not today considered targets.

So the first choice to make here is, do we:

1) Allow dependencies to display these even though they are not targets or 2) Represent transitive requirements from a lockfile as targets, of a special type, and (optionally) read them into the build graph like any other target

I lean towards 2, because it would make everything else work in a natural way (e.g., dependees and paths would work as well). But I slightly fear it for the same reason...

[kaos]

I like 2. and think that #17347 may be implemented such that it lays the ground work for getting closer to such a world.

[SergeBakharev]

I don't have a strong view either way, and happy to follow your lead. Seems like 1. could cause more issues down the line since these are not "real" targets.

However I can see the risk of user confusion being mitigated if the transitive requirements are only shown when a special flag is used and not shown by default.

[Eric-Arellano]

I agree with this ticket. I think it's a dupe of https://github.com/pantsbuild/pants/issues/12733, so let's track there. Thanks for bringing this issue up!