Dependabot is not updating package-lock when bumping dependencies in a monorepo

[Th3S4mur41]

Is there an existing issue for this?

• [X] I have searched the existing issues

Package ecosystem

npm

Package manager version

npm 8.19.3

Language version

node 16.19.0

Manifest location and content before the Dependabot update

package.json

{
    "name": "@atos-parallel/web",
    "private": "true",
    "engines": {
        "node": ">= 16.0.0",
        "npm": ">= 8.0.0"
    },
    "workspaces": [
        "packages/docs"
    ],
    "scripts": {
    },
    "devDependencies": {
        "@atos-parallel/release-config": "^1.2.3",
        "@atos-parallel/stylelint-config": "^1.2.0",
        "@babel/core": "^7.20.5",
        "@commitlint/cli": "^17.3.0",
        "@commitlint/config-conventional": "^17.3.0",
        "@percy/cli": "^1.16.0",
        "@typescript-eslint/eslint-plugin": "^5.47.0",
        "@typescript-eslint/parser": "^5.47.0",
        "autoprefixer": "^10.4.13",
        "concurrently": "^7.6.0",
        "cssnano": "^5.1.14",
        "eslint": "^8.30.0",
        "hint": "^7.1.3",
        "husky": "^8.0.2",
        "lint-staged": "^13.1.0",
        "postcss": "^8.4.20",
        "postcss-nesting": "^10.2.0",
        "postcss-preset-env": "^7.8.3",
        "prettier": "^2.8.1",
        "pretty-quick": "^3.1.0",
        "rimraf": "^3.0.2",
        "typedoc": "^0.23.23",
        "typedoc-plugin-markdown": "^3.14.0"
    }
}

/packages/docs/package.json

{
    "name": "@atos-parallel/docs",
    "version": "1.4.0",
    "private": true,
    "scripts": {
    },
    "dependencies": {
        "@atos-parallel/icons": "^1.2.12",
        "lit": "^2.5.0"
    },
    "devDependencies": {
        "@babel/core": "^7.20.5",
        "@percy/storybook": "^4.3.4",
        "@storybook/addon-a11y": "^6.5.14",
        "@storybook/addon-actions": "6.5.14",
        "@storybook/addon-docs": "6.5.14",
        "@storybook/addon-essentials": "6.5.14",
        "@storybook/addon-links": "6.5.14",
        "@storybook/addon-postcss": "^2.0.0",
        "@storybook/addons": "6.5.14",
        "@storybook/builder-vite": "^0.2.6",
        "@storybook/theming": "6.5.14",
        "@storybook/web-components": "6.5.14",
        "babel-loader": "^9.1.0",
        "postcss": "^8.4.20",
        "storybook-addon-designs": "^6.3.1"
    },
    "peerDependencies": {
        "@atos-parallel/components": "1.1.1"
    }
}

dependabot.yml content

version: 2

registries:
  github:
    type: npm-registry
    url: https://npm.pkg.github.com
    token: ${{secrets.DEPENDABOT_TOKEN}}

updates:
  - package-ecosystem: 'npm'
    directory: '/' # Location of package manifests
    registries: '*'
    schedule:
      interval: 'daily'
      time: '00:00'
      timezone: 'Europe/Berlin'

Updated dependency

@storybook/builder-vite from 0.2.5 to 0.2.6

What you expected to see, versus what you actually saw

Dependabot should update @storybook/builder-vite to version 0.2.6 in package.json in the 'docs' workspace and the package-lock.json in the root of the monorepo. The package-lock.json is not updated though

Native package manager behavior

package-lock.json in the root of the monorepo is updated too.

Images of the diff or a link to the PR, issue, or logs

https://github.com/atos-parallel/web/pull/699

Smallest manifest that reproduces the issue

No response

[deivid-rodriguez]

@Th3S4mur41 Would it be possible for you to create an open source repo with this problem? That would make it easier to fix.

[Th3S4mur41]

@deivid-rodriguez I tried to reproduce the issue in another monorepo Unfortunately, in this case it worked as expected.

Is there anything else I can provide to help you investigate what might have been a glitch?

[Th3S4mur41]

Facing the issue again in the same repo with the next update of same dependency 🤔

I tried dependabot recreate but the outcome is the same...

Running npm i on the dependabot branch in my codespaces makes the update to package-lock.json

Same update on the demo monorepo went through and updated both 😒

[deivid-rodriguez]

I have little experience with npm so my intuition in this ecosystem is low. But I guess I would try figuring out the differences between the demo monorepo. For example, it seems that your public demo does not use registries in dependabot configuration, could that be related?

[Th3S4mur41]

Good point, let me try to add that to the demo. However, this is not a global issue... All other dependencies in the problem repo are updated by dependabot just fine. So far this happend twice, each time only with the one dependency

[Th3S4mur41]

@deivid-rodriguez demo repo updated to reflect also this aspect:

[Th3S4mur41]

Even with the added dependencies from a different registry and dependabot also using GitHub package registries, the issue is still not reproducible. bump @storybook/builder-vite from 0.2.7 to 0.3.0 worked fine in the demo repo, but still missed to update the package lock in the private repo 🤔

[deivid-rodriguez]

That's too bad 😢. There must be something different between those setups that's triggering the issue, we need to figure out what that is 🤔.

[david-zarandi-old]

I ran into the same issue, here is my repository: https://github.com/azuwey/supreme-memory

A potential solution could be to specify the lock file location, because it is different than the package.json file's location, because I think the problem is that dependabot doesn't find the lock file in the project.

[Th3S4mur41]

Right, the jsdom PR looks very similar to what I'm seeing. @deivid-rodriguez is there a way to pull logs or something that would be helpful?

[deivid-rodriguez]

Now that we have open source repositories reproducing this, it should be easier to figure out. In any case you should be able to see logs at the "Insights > Dependency Graph > Dependabot" page.

[david-zarandi-old]

@deivid-rodriguez So I checked it, and it seems that, it doesn't detect the dependencies in the package-lock.json, Insights > Dependency Graph > Dependencies > package-lock.json says: package-lock.json has no dependencies or is too large to display, under the Dependabot tab I can see the package-lock.json by clicking on the three dot next to the package.json label (Monitored dependency files).

What should I look for in the logs?

[deivid-rodriguez]

I'm not sure, I tried this locally and the logs don't tell me much. But at least I can reproduce things locally now.

[Th3S4mur41]

This time it happend in my demo repo too

[svenjacobs]

I'm facing the same issue. Unfortunately this is a commercial project so I can't share any code.

We have the following monorepo structure utilizing npm workspaces

project-1/package.json
project-2/package.json
package.json
package-lock.json

so there is only one package-lock.json at the root of the repo.

This is our .github/dependabot.yml:

version: 2
updates:
    - package-ecosystem: 'npm'
      directory: 'project-1'
      schedule:
          interval: 'weekly'
          day: 'monday'
          time: '09:00'
          timezone: 'Europe/Berlin'
      versioning-strategy: increase
      labels:
          - 'dependencies'
      ignore:
          - dependency-name: '@types/node'
      assignees:
          - 'dev1'
          - 'dev2'

    - package-ecosystem: 'npm'
      directory: 'project-2'
      schedule:
          interval: 'weekly'
          day: 'monday'
          time: '09:00'
          timezone: 'Europe/Berlin'
      versioning-strategy: increase
      labels:
          - 'dependencies'
      assignees:
          - 'dev1'
          - 'dev2'

I think it would help if Dependabot performs a npm install --package-lock-only after modifying any package.json and adding the modified package-lock.json to the commit afterwards.

Update:

Changing from multiple configs for individual projects to a single config with directory: '/' as described here fixes the problem for me. However this also means that currently no individual Dependabot configuration can be set for monorepo projects.

[SpiritusDeos]

Is someone success to setup dependabot with a monorepo and to update the correct yarn.lock (regarding which directory the package is updated) ?

[Develliot]

I have a monorepo using NPM workspaces, with a similar set up to @svenjacobs but with 4 packages. My dependabot.yml targets the route directory directory: '/' for NPM updates. Some workspaces have no issues, when a dependencies are bumped the in a workspace package.json file the shared repo lockfile gets bumped too. However with 4th and last listed workspace dependabot bumps the deps in the workspace package.json but the shared repo lockfile never updates. I can't see any obvious differences in the workspace package.json files. All workspaces were created with the npm init -w command. In the shared package.json file I can see all workspaces listed in the same manor. At the moment I'm not sure why dependabot has an issue with the last work space.

[Snailedlt]

Happens to me too here: https://github.com/Snailedlt/Markdown-Videos/pull/132

Though I'm using pnpm instead of npm

[logseq-cldwalker]

@deivid-rodriguez Hi. We're also experiencing this on a public, monorepo project. We don't have a dependabot.yml. https://github.com/logseq/logseq/pull/10079 is an example of a PR that didn't update yarn.lock

[deivid-rodriguez]

Thank you. This issue is actionable now since several repros have been shared. Only thing needed is someone to work on it.

We have not prioritized this internally yet but anybody interested on a fix is welcome to get the repo setup for development and investigate.

[tunessofia]

i have been also experiencing this issue in a project where we currently upgraded to node 20.x. does this might be due to dependabot using node 16 and the lock file is generated with another npm version? and cannot upgrade it? looking at the logs it doesnt display any errors or warnings

[DerTimonius]

I looked into this a bit, since we are facing this issue on a closed source project. I noticed that the updates of the lockfiles are happening if one of the following conditions are true:

• the package is found in the root packagen.json
• the package is used in multiple subdirectories of the monorepo

If the package is found in something like apps/web/package.json the lockfile is not updated.

I would be open to help with this issue, but I never used Ruby before. So the members might have to be patient with me 🙂

[deivid-rodriguez]

That would be fantastic @DerTimonius, thanks so much!

First step would be to make sure the issue is reproducible either with our CLI or our dry-run.rb script. See https://github.com/dependabot/dependabot-core#debugging-problems. Public repositories exposing the issue have been shared previously, so better use those to leave authentication issues out of the way.

[DerTimonius]

@deivid-rodriguez I tried to setup the dev environment for npm_and_yarn but running bin/docker-dev-shell npm_and_yarn/ gave me the following error:

I guess there may be an issue with the Dockerfile here, but what should be the correct line? https://github.com/dependabot/dependabot-core/blob/ea7ad41e384f2e760d2b956cbd881c7afd10a35e/npm_and_yarn/Dockerfile#L56

[deivid-rodriguez]

Can you remove the trailing slash from your command? Run bin/docker-dev-shell npm_and_yarn rather than bin/docker-dev-shell npm_and_yarn/. I think that should fix it!

[DerTimonius]

So, I ran the dry-run script on the provided monorepos, but it only recognized the root package.json file, it did not go deeper

[deivid-rodriguez]

Cool. Can you show how workspaces are setup?