Conflict in choosing the manager to process lockfile when multiple managers are using the same package file

[RahulGautamSingh]

Describe the proposed change(s).

When multiple managers work on the same package file, the lockfile maintenance breaks because all lockfiles dependent on the same package file are updated with only one manager instead of their corresponding managers.

Related discussion: https://github.com/renovatebot/renovate/discussions/26851 Detailed explanation: https://github.com/renovatebot/renovate/discussions/26851#discussioncomment-8264904

[viceice]

I saw this happen too when the lockfile no longer has changes after rebase. PR should be auto-closed when this happens.

[nejch]

I think it's not only an issue with lockfile maintenance but also normal grouped dependencies that deal with lockfiles, like the pep621 vs. poetry collision described in the discussion (maybe you already meant that, but just wanted to confirm since you used the term lockfile maintenance above). We observed a similar timeline as in the other comment (this started happening mid January).

[fgreinacher]

Feel free to assign this to me, I'll have a look

[viceice]

this should probably be handled like the bun manager, which is overriding the npm manager. so npm, yarn and pnpm are automatically disabled when a bun lockfile is found.

[fgreinacher]

this should probably be handled like the bun manager, which is overriding the npm manager. so npm, yarn and pnpm are automatically disabled when a bun lockfile is found.

The pep621 manager updates different dependencies than the poetry manager. So I'd say overriding it would be incorrect here. It's a similar situation like with gitlabci and gitlabci-include, but there we don't have lockfiles :)

[rarkins]

So is there two lock files to maintain for one package file?

[nejch]

this should probably be handled like the bun manager, which is overriding the npm manager. so npm, yarn and pnpm are automatically disabled when a bun lockfile is found.

The pep621 manager updates different dependencies than the poetry manager. So I'd say overriding it would be incorrect here. It's a similar situation like with gitlabci and gitlabci-include, but there we don't have lockfiles :)

I agree with @fgreinacher as this currently seems to happen because of build-system dependencies, but tools like PDM could be using PEP621 in combination with lockfiles (https://pdm-project.org/latest/usage/hooks/#dependencies-management),. But I can't see that happening at the same time in a single project.

[fgreinacher]

Yes, it's a bit weird right now. The PEP621 manager should probably not do anything when Poetry is used. Or it should have full Poetry support, but that would be quite some effort it seems.

Downside of disabling it in that case would be that we'd lost build-system updates for Poetry

[fgreinacher]

So is there two lock files to maintain for one package file?

No, the problem here is that we have multiple managers with artifact/lockfile support for a single package file.

[nejch]

One solution for build-system IMO would be to make it a pep518 manager, which would actually be more accurate:

https://peps.python.org/pep-0518/

The [build-system] table is used to store build-related data. Initially only one key of the table will be valid and is mandatory for the table: requires. This key must have a value of a list of strings representing PEP 508 dependencies required to execute the build system (currently that means what dependencies are required to execute a setup.py file).

The mechanism is almost the same I think so it could inherit if it's feasible. WDYT?

[fgreinacher]

Or maybe just a pyproject manager, that handles everything described in https://packaging.python.org/en/latest/specifications/pyproject-toml/#pyproject-toml-spec?

[viceice]

this should probably be handled like the bun manager, which is overriding the npm manager. so npm, yarn and pnpm are automatically disabled when a bun lockfile is found.

The pep621 manager updates different dependencies than the poetry manager. So I'd say overriding it would be incorrect here. It's a similar situation like with gitlabci and gitlabci-include, but there we don't have lockfiles :)

no, it's not. it was a fault to add gitlabci-include as separate manager. that will soon be merged with gitlabci manager.

[fgreinacher]

no, it's not. it was a fault to add gitlabci-include as separate manager. that will soon be merged with gitlabci manager.

I agree, just saying that we have a similar situation here ;)

[fgreinacher]

Independent of the discussion how the managers should be structured, and assuming Renovate should in general support multiple managers per package file, here is a proposal to handle this properly: https://github.com/renovatebot/renovate/pull/27016.

[secustor]

The pep621 was designed around that we will eventually migrate poetry to a sub processor of it, tough I had not reason yet to tackle this.

[renovate-release]

:tada: This issue has been resolved in version 37.174.5 :tada:

The release is available on:

• GitHub release
• 37.174.5

Your semantic-release bot :package::rocket: