search

monperrus / crawler-user-agents

Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star:
MIT License
1.13k stars 242 forks source link

Add npm dependencies with lockfile #351

Closed ericcornelissen closed 2 months ago

ericcornelissen commented 3 months ago

Add pacote (at v11.1.11) as a proper npm dependency because it was being installed in CI dynamically. It is added as a devDependency because users of the package do not need it so shouldn't be included for them. Accordingly, update CI pipelines to use npm clean-install, which will install exactly what is in the lockfile, for more details see: https://docs.npmjs.com/cli/v10/commands/npm-ci.

This change hardens the supply chain. Before, even if you trust npm to be immutable and always give you the same code for pacote, it would resolve different versions of transitive dependencies (evidenced by the presence of version ranges in the dependency list for pacote in the newly added lockfile, line 1294 to 1314). Now, both direct and transitive dependencies are pinned, and their checksum can be checked at install time to ensure you're getting the same bits as you did last time.

Additionally, you could consider adding an .npmrc file with ignore-scripts=true to avoid install time attacks (it looks like neither this project nor its dependencies use any scripts).

monperrus commented 2 months ago

Thanks @ericcornelissen

Now we start to need renovate here, WDYT?

ericcornelissen commented 2 months ago

Now we start to need renovate here, WDYT?

That would make some sense @monperrus (or Dependabot). If you do decide to use a dependency maintenance bot you could consider also enabling it for your GitHub Actions (e.g. with renovate).

monperrus commented 2 months ago

Dependabot is working by default: https://github.com/monperrus/crawler-user-agents/pull/352

ericcornelissen commented 2 months ago

Given the absence of a dependabot config file, I think #352 was a dependabot security update (however, that's only visible to users with the right authorization). Security updates are indeed enabled by default, but "regular" dependency maintenance is an opt-in feature.