Non-reproducible package-lock.json generated

bnavigator commented 1 month ago

As reported by @bmwiedemann in boo#1231254, the instructions as of now cause the creation of non-reproducible package-lock.json files.

Specfile of python-panel:

# package-lock.json file generated with procedure:
# - delete old package-lock.json in panel subdirectory
# - add '"typescript": "^4.2.0"' to package.json devDependencies
# - npm install --package-lock-only --legacy-peer-deps --ignore-scripts
Source10:       package-lock.json
# node_modules generated using "osc service mr" with https://github.com/openSUSE/obs-service-node_modules
Source11:       node_modules.spec.inc
BuildRequires:  local-npm-registry

%prep
pushd panel
rm package-lock.json
local-npm-registry %{_sourcedir} install --include=dev --include=peer
popd

After the %build stage, there is a panel/package-lock.json again, which differs from %{SOURCE10}:

abuild@skylab:~> diff -u rpmbuild/SOURCES/package-lock.json rpmbuild/BUILD/panel-1.5.0/panel/package-lock.json | head -20
--- rpmbuild/SOURCES/package-lock.json  2024-09-25 15:21:29.000000000 +0000
+++ rpmbuild/BUILD/panel-1.5.0/panel/package-lock.json  2024-10-02 19:52:19.444156444 +0000
@@ -26,13 +26,12 @@
         "@typescript-eslint/parser": "^7.2.0",
         "acorn": "^8.11.3",
         "eslint": "^8.57.0",
-        "flatpickr": "^4.6.9",
-        "typescript": "^4.2.0"
+        "flatpickr": "^4.6.9"
       }
     },
     "node_modules/@babel/runtime": {
       "version": "7.25.6",
-      "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.25.6.tgz",
+      "resolved": "http://localhost:42473/-/@babel-runtime-7.25.6.tgz",
       "integrity": "sha512-VBj9MYyDb9tuLq7yzqjgzt6Q+IBQLrGZfdjOekyEirZPHxXWoTSGUTMrpsfi58Up73d13NfYLv8HT9vmznjzhQ==",
       "license": "MIT",
       "dependencies": {
@@ -44,7 +43,7 @@
     },
...

Of I use --no-package-lock (https://github.com/openSUSE/npm-localhost-proxy/issues/1), the file will be created by the npm call of the python wheel build later on. With the same localhost problem.

This is avoided with

Index: python-panel.spec
===================================================================
--- python-panel.spec   (revision 6c8acd1f4122bca11fe7842e42cba65f)
+++ python-panel.spec   (working copy)
@@ -136,7 +136,7 @@
 sed -i 's|bundle_resources()$|assert os.path.exists("panel/dist/bundled/font-awesome")|' hatch_build.py
 # npm registry for Source10 provided in Source11
 pushd panel
-rm package-lock.json
+cp %{SOURCE10} ./
 local-npm-registry %{_sourcedir} install --include=dev --include=peer
 popd
 sed -i /asyncio_default_fixture_loop_scope/d pyproject.toml

Which results in a deterministic package-lock.json

abuild@skylab:~> diff -u rpmbuild/SOURCES/package-lock.json rpmbuild/BUILDROOT/python-panel-1.5.0-0.x86_64/usr/lib/python3.11/site-packages/panel/package-lock.json
--- rpmbuild/SOURCES/package-lock.json  2024-09-25 15:21:29.000000000 +0000
+++ rpmbuild/BUILDROOT/python-panel-1.5.0-0.x86_64/usr/lib/python3.11/site-packages/panel/package-lock.json     2024-10-02 19:58:31.089170061 +0000
@@ -26,8 +26,7 @@
         "@typescript-eslint/parser": "^7.2.0",
         "acorn": "^8.11.3",
         "eslint": "^8.57.0",
-        "flatpickr": "^4.6.9",
-        "typescript": "^4.2.0"
+        "flatpickr": "^4.6.9"
       }
     },
     "node_modules/@babel/runtime": {
@@ -2729,6 +2728,7 @@
       "integrity": "sha512-1FXk9E2Hm+QzZQ7z+McJiHL4NW1F2EzMu9Nq9i3zAaGqibafqYwCVU6WyWAuyQRRzOlxou8xZSyXLEN8oKj24g==",
       "dev": true,
       "license": "Apache-2.0",
+      "peer": true,
       "bin": {
         "tsc": "bin/tsc",
         "tsserver": "bin/tsserver"

Conclusion: The instructions in the README must be adjusted from not only deleting the original package-lock.json but replacing it with our own.

https://github.com/openSUSE/obs-service-node_modules/blob/1daa6a2701acc862dd051e00a131693d7640f189/README.md?plain=1#L31-L33

bnavigator commented 1 month ago

Its' even worse. What is described as fix above only works in a local osc build. On the server I now get resolver errors. :(

AdamMajer commented 1 month ago

A few things here. First, you are patching package.json and generating package-lock.json without keeping the patch? Why? package.json patch should be part of the package sources.

Secondly, package-lock.json is a generated file and you can view it as part of the intermediate sources generated. The package-lock that is generated in the chroot/VM will be different when run vs. remote npm registry since it contains resolved FQDN. That's just how npm works.

Finally, the package-lock.json generated by npm to be used by the node_modules service is for one purpose only -- to download the remote assets so they are available in the chroot later on. This OBS service downloads the resolved tarballs for the build service.

The one thing that could be added to instructions is to have rm -f package-lock.json to be run before the install command is run. What do you think?

openSUSE / obs-service-node_modules

Non-reproducible package-lock.json generated #41