[RRFC] Support installing from only a lock file

[gingermusketeer]

Motivation ("The Why")

When developing in a docker environment, rebuilding images and installing dependencies can be quite slow. Using best practices it is recommended to have a docker file with the following steps:

1. Copy package.json and package-lock.json files into image
2. Install dependencies
3. Copy rest of application

Doing it this way ensure that we can skip step 2 if nothing has changed in step 1. Ideally this means that step 1 should only contain the minimum to install the dependencies in step 2.

For npm v9 the minimum needed to be able to install dependencies in a production like setting such as a docker file is both the package.json and package-lock.json.

Relying on package.json has some downsides because it is used for a lot of other configuration:

• npm scripts
• eslint configuration
• jest configuration
• repository and licence information.

If any of these change then step 2 of the docker build cannot be skipped.

Example

• Solving this would speed up docker builds for people where package.json has changed but package-lock.json has not changed.

How

Current Behaviour

• Currently both package.json and package-lock.json is needed to be able to install packages.

Trying to install with only a package-lock.json file present results in the following error:

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /path/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/path/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

Desired Behaviour

• npm install should support installing from only a package-lock.json
• npm ci should support installing from only a package-lock.json.

References

• Potentially related RRFC https://github.com/npm/rfcs/issues/415
• Recommended docker file for node

[ljharb]

A lockfile doesn't have sufficient information. Why would it be valuable to duplicate the information in package.json into the lockfile?

It seems like a smarter diffing algorithm would help you here - for example, don't just diff the files, but delete the keys you don't care about, sort the remaining top-level keys, and then diff them.

[gingermusketeer]

A smarter diffing algorithm would definitely be an alternative. It does make it more complicated however as it would necessitate a tool being installed or copy pasted between projects.

When you say that the information in a lock file doesn't have sufficient information, what is missing? My understanding was that lock files had everything necessary.

As of npm v7, lockfiles include enough information to gain a complete picture of the package tree, reducing the need to read package.json files, and allowing for significant performance improvements.

source

This suggests the need for a package.json are reduced so reducing further might be possible.

Thanks for taking the time to read the proposal!

[ljharb]

reducing, not eliminating, is what i read from that.

[gingermusketeer]

After a big more learning and experimenting related to this I have a work around which involves:

1. Generating a package-docker.json file on postinstall of the package.json
2. Copying package-docker.json in place of package.json prior to the dependency install step.
3. Installing dependencies
4. Replacing minimal package.json with actual file so npm scripts etc are available.

Code

The `postinstall` step is: ```json "postinstall": "node -e \"require('node:fs').writeFileSync( 'package-docker.json', JSON.stringify({...require('./package-lock.json').packages[''], overrides: require('./package.json').overrides}, null, 2) );\"" ``` In a more readable format: ```js require("node:fs").writeFileSync( "package-docker.json", JSON.stringify( { ...require("./package-lock.json").packages[""], overrides: require("./package.json").overrides, }, null, 2 ) ); ```

For step 1 I was able to generate this completely from the package-lock.json file except for overrides so it looks like most of the information needed is in the package-lock.json.

The problem with this workaround is that it will most likely break tools like dependabot, renovate or any other tool which only expects package-lock.json to change when package.json changes.