dependabot / dependabot-core

πŸ€– Dependabot's core logic for creating update PRs.
https://docs.github.com/en/code-security/dependabot
MIT License
4.69k stars 1.02k forks source link

Feature request: support the `pnpm` package manager #1736

Closed jbergstroem closed 1 year ago

jbergstroem commented 4 years ago

pnpm is an alternative to npm and yarn that has been around a fairly long time. It has its own lockfile in yaml and should be relatively straightforward to support. Would PRs/other types of help be accepted or are adding more pr's a "can of worms"?


[Edit] April 2022 - since the thread is growing: there is still no update on when this will be implemented. If you need this now, your best bet is to use renovatebot: https://docs.renovatebot.com/javascript/

[Edit] November 2022 - for now, github staff suggests using a workaround using the dependency submission api

jhutchings1 commented 1 year ago

πŸ‘‹πŸ» Hi folks, I appreciate everyone's patience on this issue as the team has been focused on other issues. This is our 3rd highest πŸ‘πŸ» 'd issue, so it's on our radar, but I do not have an ETA to share at this time.

I did want to share an update which I hope you'll find helpful, however, which is that you can now get Dependabot alerts for pnpm using GitHub Actions and the new(ish) dependency submission API. You'll find a working workflow example you can use here: https://github.com/jhutchings1/pnpm-sample/blob/main/.github/workflows/sbom.yml.

Anyone had any luck implementing this? My github action is successful but can't see any snapshot dependecies on the repo.



Notice: Snapshot successfully created at 2022-11-16T02:58:00.770Z

Is the dependency graph enabled? Happy to take a look if it's a public repo.

jinlinux commented 1 year ago

πŸ‘‹πŸ» Hi folks, I appreciate everyone's patience on this issue as the team has been focused on other issues. This is our 3rd highest πŸ‘πŸ» 'd issue, so it's on our radar, but I do not have an ETA to share at this time.

I did want to share an update which I hope you'll find helpful, however, which is that you can now get Dependabot alerts for pnpm using GitHub Actions and the new(ish) dependency submission API. You'll find a working workflow example you can use here: https://github.com/jhutchings1/pnpm-sample/blob/main/.github/workflows/sbom.yml.

Anyone had any luck implementing this? My github action is successful but can't see any snapshot dependecies on the repo.


Notice: Snapshot successfully created at 2022-11-16T02:58:00.770Z

Is the dependency graph enabled? Happy to take a look if it's a public repo.

Thanks unfortunately it is a private repo.

Dependency graph is enabled and there are dependencies being picked up from package.json. Not sure where to find the uploaded dependencies or how to troubleshoot it though.

jhutchings1 commented 1 year ago

πŸ‘‹πŸ» Hi folks, I appreciate everyone's patience on this issue as the team has been focused on other issues. This is our 3rd highest πŸ‘πŸ» 'd issue, so it's on our radar, but I do not have an ETA to share at this time.

I did want to share an update which I hope you'll find helpful, however, which is that you can now get Dependabot alerts for pnpm using GitHub Actions and the new(ish) dependency submission API. You'll find a working workflow example you can use here: https://github.com/jhutchings1/pnpm-sample/blob/main/.github/workflows/sbom.yml.

Anyone had any luck implementing this? My github action is successful but can't see any snapshot dependecies on the repo.


Notice: Snapshot successfully created at 2022-11-16T02:58:00.770Z

Is the dependency graph enabled? Happy to take a look if it's a public repo.

Thanks unfortunately it is a private repo.

Dependency graph is enabled and there are dependencies being picked up from package.json.

Not sure where to find the uploaded dependencies or how to troubleshoot it though.

Thanks for confirming. It could be an issue with the API. Do you mind opening a support ticket so we can dig into the details privately?

coodyme commented 1 year ago

πŸ‘‹πŸ» Hi folks, I appreciate everyone's patience on this issue as the team has been focused on other issues. This is our 3rd highest πŸ‘πŸ» 'd issue, so it's on our radar, but I do not have an ETA to share at this time.

I did want to share an update which I hope you'll find helpful, however, which is that you can now get Dependabot alerts for pnpm using GitHub Actions and the new(ish) dependency submission API. You'll find a working workflow example you can use here: https://github.com/jhutchings1/pnpm-sample/blob/main/.github/workflows/sbom.yml.

I tried that solution but I'm getting this error on organization. Works well in personal repository...

image

jonathborg commented 1 year ago

I have fixed the error purl is missing the required "name" component error when the organization name starts with an @. This is my version of the SPDX to dependency graph action. And this is the main change that fixes the error.

borstelmannl commented 1 year ago

Simple solution for pnpm still needed!

ibqn commented 1 year ago

It would be great to see an official support of pnpm for dependabot!

JUNNNI commented 1 year ago

Any update about including pnpm support?

jhutchings1 commented 1 year ago

Any update about including pnpm support?

Not at this time. We're focused on other high priority requests for this quarter. If you want alerts for your pnpm dependencies, check out these instructions. We'll post to this issue when there're more to share.

levrik commented 1 year ago

Due to this I postponed the move to PNPM multiple times now. Yesterday I made the decision to just move on and switch to Renovate. Besides supporting PNPM it also supports grouped updates which is another long-standing issue with Dependabot.

jhutchings1 commented 1 year ago

Due to this I postponed the move to PNPM multiple times now. Yesterday I made the decision to just move on and switch to Renovate. Besides supporting PNPM it also supports grouped updates which is another long-standing issue with Dependabot.

I know this is a long-standing request and it's a big deal for many of you. I appreciate the patience you've all given us, but understand that you've got to do what's best for your project. I hope we can win you back in the future.

MartinX3 commented 1 year ago

I don't understand the massively slow development of dependabot. At least Microsoft is supporting this project or it won't be integrated into the GitHub software.

levrik commented 1 year ago

@MartinX3 Yeah. Dependabot is moving horribly slow. I have the feeling that it is kinda on the back burner for MS/GitHub. Which is quite sad as I feel much more confident letting GitHub itself handle the dependency updates instead of giving a 3rd party access.

Since the core updating logic of Dependabot is open source, probably someone from the community could contribute PNPM support, I would even try doing it... if I would have the time because I imagine this is something that takes longer than a single afternoon.

ssbarnea commented 1 year ago

With the risk of sounding paranoiac, we cannot really forget about the extinguish part of EEE and that would explain at least few things around:

levrik commented 1 year ago

@ssbarnea Since MS is using pnpm themselves a lot I doubt that they want to extinguish anything here. Also MS has changed a lot since Nadella took over.

See https://github.com/dependabot/dependabot-core/issues/1736#issuecomment-1098603664

Also a lot of other parts of GitHub support pnpm. I would suggest you to read through the whole issue to get the full picture.

ssbarnea commented 1 year ago

@levrik Please show me one vscode extension maintained by microsoft that uses pnpm (in fact even maintained by someone else). I did search a lot and failed to find one to use as reference basically because I wanted to use pnpm myself. Even finding ones using yarn is hard, npm being far more popular. I often find issues due to lack of good support for (current versions of) yarn (not v1).

I hope someone from this thread would spend the extra time and build an action replaces dependabot version updates, as I thin that is more likely to happen than seeing dependabot itself getting it. I am worried that by the time it would get support for it, we will have another emerging package manager that aims to obsolete the other ones, basically never reaching a point where dependabot supports modern build toolset.

PS. The "eee" was more like a joke, kind-of.

BleedingDev commented 1 year ago

Prisma is using PNPM. That's why my project https://github.com/pegak/nauc-me-it migrated. We struggled with yarn berry, but now with pnpm almost everything is smooth (except dependabot integration).

karlAlnebratt commented 1 year ago

Are there plans for supporting pnpm?

velrest commented 1 year ago

@karlAlnebratt

Any update about including pnpm support?

Not at this time. We're focused on other high priority requests for this quarter. If you want alerts for your pnpm dependencies, check out these instructions. We'll post to this issue when there're more to share.

AndrewCraswell commented 1 year ago

I know back in October this was the 3rd most upvoted issue. But since then this is now the 2nd most upvoted. When tallying total reactions, it's behind by only 5 reacts. Seems pnpm has had a surge of popularity over the past year, and at Microsoft it's pretty common to see it used.

Dependabot would pair really well with internal tools like Component Governance and would go a long way towards improving maintainability and compliance. Having used Dependabot in Github repos a lot, it's one of the things I miss the most in our Azure DevOps repositories.

We tried using Renovate but couldn't get it working within our DevOps pipeline agents. I'm not eager to shift away from pnpm, so it's down to a waiting game. Looking forward to future updates on the timing of this.

cristobal commented 1 year ago

I know back in October this was the 3rd most upvoted issue. But since then this is now the 2nd most upvoted. When tallying total reactions, it's behind by only 5 reacts. Seems pnpm has had a surge of popularity over the past year, and at Microsoft it's pretty common to see it used.

Dependabot would pair really well with internal tools like Component Governance and would go a long way towards improving maintainability and compliance. Having used Dependabot in Github repos a lot, it's one of the things I miss the most in our Azure DevOps repositories.

We tried using Renovate but couldn't get it working within our DevOps pipeline agents. I'm not eager to shift away from pnpm, so it's down to a waiting game. Looking forward to future updates on the timing of this.

In our company almost every project that has switched to pnpm have started using renovate instead, most of us would prefer to stick to one tooling for updates but must say that renovate is a good better than alternative than dependabot for npm dependencies and has improved a lot over the couple of years. Can gladly recommend it as an alternative to dependabot πŸ‘‹πŸ‘

No disrespect to dependabot but seems to have staggered a bit due to priorities, best of both worlds would be if both renovate and dependabot teamed up and merged into one project πŸ™πŸ½. There are still some features I miss from dependabot since it’s integrated into Github however the ability to finetune in renovate is betterπŸ‘

osrl commented 1 year ago

Happy 3rd anniversary to this issue πŸŽ‰

morewings commented 1 year ago

This thread is a best ad for Renovate. You’ve forgotten Asimov’s laws, @dependabot

leemeichin commented 1 year ago

rather than supporting pnpm or any other package manager specifically, is it not an option to defer to corepack for NPM dependencies instead? It's in the 18.x LTS. It's been mentioned before here, in 2021. I was confused why dependabot doesn't run on one of our NextJS repos until I realised we use pnpm

could I just run dependabot as a scheduled GH Action instead? πŸ˜…

(πŸ‘ŽπŸΌ for marketing that other tool here - want our problem to be solved not sold)

morewings commented 1 year ago

(πŸ‘ŽπŸΌ for marketing that other tool here - want our problem to be solved not sold)

It's a free (like speech and beer) market and at this moment Renovate has much better offer. I don't see any reason why I shouldn't speak about that.

lotyp commented 1 year ago

Just forget about dependabot and use renovate in the cloud or self-hosted. It is a little bit complicated to configure, but it needs to be done once

deivid-rodriguez commented 1 year ago

Hello πŸ‘‹ πŸ‘‹

We are getting started with looking into pnpm support.

Feel free to share your public repositories using pnpm and describe anything relevant to your setups, for example:

This will help us better understand the different use cases and the most common expectations when upgrading dependencies with pnpm.

Thanks!

dbartholomae commented 1 year ago

Not sure if it's an ideal first use case, but pnpm is a common package manager for monorepos. Here's an example: https://github.com/startup-cto/blog

The setup is similar to some private company repos that I'm aware of.

osrl commented 1 year ago

hey @deivid-rodriguez , it's great news. We were planning to switch over but I think we can wait. Can you give us an ETA?

wilhen01 commented 1 year ago

@deivid-rodriguez we use pnpm to manage a monorepo. Ours isn't public, but you can find them quite easily by searching for the pnpm-workspace.yaml configuration file across github: https://github.com/search?q=pnpm-workspace.yaml&type=code.

This appears to be a fairly standard example but there are definitely others: https://github.com/alibaba/ice

deivid-rodriguez commented 1 year ago

@osrl There's no fixed ETA but this a a priority for us during this coming quarter.

Thanks @wilhen01! I can see pnpm is very popular for monorepos and found this very useful source of examples: https://pnpm.io/workspaces#usage-examples.

Stanzilla commented 1 year ago

Hello πŸ‘‹ πŸ‘‹

We are getting started with looking into pnpm support.

Feel free to share your public repositories using pnpm and describe anything relevant to your setups, for example:

  • Any specific configurations that you use.
  • Pnpm version & lockfile format.
  • How do you update dependencies locally?

This will help us better understand the different use cases and the most common expectations when upgrading dependencies with pnpm.

Thanks!

Here is a very popular repo that uses it https://github.com/vercel/next.js

sndrs commented 1 year ago

we use it in an Nx monorepo (using pnpm workspaces) here https://github.com/guardian/csnx

raulfdm commented 1 year ago

Hello πŸ‘‹ πŸ‘‹

We are getting started with looking into pnpm support.

Feel free to share your public repositories using pnpm and describe anything relevant to your setups, for example:

  • Any specific configurations that you use.
  • Pnpm version & lockfile format.
  • How do you update dependencies locally?

This will help us better understand the different use cases and the most common expectations when upgrading dependencies with pnpm.

Thanks!

Ins't a huge repo but I've been working on this for a couple of years already: https://github.com/raulfdm/raulmelo-studio

I had to move away from dependabot when I migrated it from yarn + lerna to pnpm workspace. There you can find the renovate config and workflow.

When I don't want to wait the bot upgrading my deps, I run manually pnpm up -i --latest -r and pick what I want to upgrade. Other than that, I wait the bot create the PR for me.

steveluscher commented 1 year ago

I consider pnpm a drop-in replacement for npm that's just faster, better, and more monorepo-friendly. My use case is β€˜npm, but if it sucked less.’

octogonz commented 1 year ago

We are getting started with looking into pnpm support.

image

image

Better late than never! 😊 Thanks so much!

Feel free to share your public repositories using pnpm and describe anything relevant to your setups, for example:

  • Any specific configurations that you use.

@deivid-rodriguez I wanted to point out that the Rush monorepo orchestrator uses PNPM as its default/recommended package manager, but with a modified configuration for the PNPM workspace. These modifications facilitate various Rush-specific enhancements that are integrated with the conventional PNPM model. The workspace definition file is relocated to common/temp/pnpm-workspace.yaml, which gets generated dynamically from the definitions in rush.json. The lockfile is stored as common/config/rush/pnpm-lock.yaml, and many settings are specified via common/config/rush/pnpm-config.json to provide better validation and policy enforcement.

Installations are performed using the rush install and rush update commands (which provide a unified interface regardless of whether the monorepo has chosen Yarn or NPM or PNPM). To access PNPM specific features, a rush-pnpm CLI wrapper is provided.

Rush also customizes many of the setting defaults for PNPM, to avoid practices that are problematic for large code bases. These differences are summarized in the Strict settings doc page for the companion tool Lockfile Explorer.

morewings commented 1 year ago

I use pnpm 7 for this project https://github.com/morewings/structure

callicles commented 1 year ago

I both use rush and turborepo with pnpm

Professionally I am working with Nike, and we also use pnpm (on GHEC) happy to point you to the internal repo if you reach out through our acount rep.

Personal public turbo: https://github.com/callicles/psychic-barnacle

elcharitas commented 1 year ago

I also make use of pnpm v8 for this project: https://github.com/elcharitas/chakra-ui-svelte

rizqirizqi commented 1 year ago

Please note that the pnpm 7 and pnpm 8 lock file is different.

vluoto commented 1 year ago

Please also note that there might not always be just one lockfile: https://pnpm.io/npmrc#shared-workspace-lockfile

deivid-rodriguez commented 1 year ago

Thanks everyone for all the examples! They're very useful for us while working on this, for example, I had not noticed the shared-workspace-lockfile setting :)

Something I should note to avoid false expectations is that we won't be supporting Rush in this ship. That's tracked separately at https://github.com/dependabot/dependabot-core/issues/2270 but I hope it will be easier once PNPM support is there.

Finally, and just to keep you all posted, I just shared an initial WIP for this feature. Not production ready at all but wanted to share some progress πŸ˜ƒ.

deivid-rodriguez commented 1 year ago

We plan to merge the PR with beta support for PNPM in the next few days. If you're interested in trying it out, let us know and we'll enable it for your projects once we merge #7081.

EDIT: In order to save issue watchers from unwanted notifications, please use https://github.com/dependabot/dependabot-core/issues/1736#issuecomment-1527947075 as a point of contact rather posting new comments on this ticket if you want to opt-in to beta testing.

steveluscher commented 1 year ago

Letting you know! https://github.com/solana-labs/solana-web3.js please!

Edit: RIP my notifications.

thmsobrmlr commented 1 year ago

Letting know as well! Repo here https://github.com/PostHog/posthog

Nfinished commented 1 year ago

All of my personal repos, thanks.

On Wed, Apr 26, 2023 at 2:08β€―PM Thomas ObermΓΌller @.***> wrote:

Letting know as well! Repo here https://github.com/PostHog/posthog

β€” Reply to this email directly, view it on GitHub https://github.com/dependabot/dependabot-core/issues/1736#issuecomment-1523845800, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAND33ZKPLBX4S5W4NC3CLDXDFQCZANCNFSM4LJTDWNA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

deivid-rodriguez commented 1 year ago

One repo each should be enough for now, since we can't easily enable the feature on all repos of an org/user. Also, they should be repositories using at least lockfileVersion: 5.4 and PNPM 7, which is what we're aiming to support right now. Thanks!

MartinX3 commented 1 year ago

You shouldn't support pnpm 7 and lockfile 5.4 and start with pnpm 8 and lockfile 6 or you will loose further users.

endigma commented 1 year ago

hussleinc/mesa would be good

deivid-rodriguez commented 1 year ago

You shouldn't support pnpm 7 and lockfile 5.4 and start with pnpm 8 and lockfile 6 or you will loose further users.

Can you clarify @MartinX3? We will support PNPM 8 and lockfileVersion 6.0, note that I said at least.