Open Andrew-Dyachenko opened 2 years ago
Ok, I am a bit confused, as these are different errors, but more importantly, different packages. The first error looks like it is for lmdb-store (different package/repo) and the second error is for node-lmdb (don't worry, I maintain lmdb-store too, I'll still help you, just want to make sure I am properly understanding where the error is). Do you know what version of lmdb-store you were trying to install in the first version? The latest version should have prebuilds for linux-x64+node-v16 (so it shouldn't need to compile at all). Also, the root error is "Operation not permitted". Is it possible that there is an existing process running with the package you are installing/upgrading over?
The second error is for node-lmdb building on mac, and looks like some mac compiler error unrelated to node-lmdb. I could add some prebuilds for macos for node-lmdb, and that would probably resolve that issue. However, I doubt that would have any effect on the first issue since it looks like it is probably unrelated.
Yes these are different errors. The first one is the original place where I have seen this at the first time. This is the GitLab's pipeline which is running on some kind of Linux in a Docker environment. Until this moment I even did not know that this project have using node-lmdb
package. I don't know what explicitness version of node-lmdb
package version GitLab's pipeline is trying to install in the first error case. I tried to find what package in this project depends on it, but didn't find it:
andrey@ru41 client % npm ls node-lmdb
main-site@1.17.0 /Users/andrey/Sites/main-site/src/client
└── (empty)
andrey@ru41 client %
Than I tried to install node-lmdb
package with command npm i node-lmdb@latest
to simulate this issue on my local machine in a macOS to get more experience about this issue. And this also caused a similar error on the local machine.
Fortunately, I managed to skip this problem and assemble the GitLab's pipeline by replacing it in Dockerfile:
RUN npm i --no-optional
to
RUN npm ci --no-optional
but this is just walkaround...
Yes please if you can do routine job like prebuilds for macOS/Linux/Windows to reduce potential assembly issues it will be wonderfull!
But the package you referenced is lmdb-store. I don't think using npm ls node-lmdb
will find lmdb-store, you need to type npm ls lmdb-store
? And updating node-lmdb won't fix your lmdb-store issue, right?
You are right!
andrey@ru41 client % npm ls lmdb-store
main-site@1.17.0 /Users/andrey/Sites/main-site/src/client
└─┬ @parcel/packager-xml@2.0.1
└─┬ @parcel/types@2.0.1
└─┬ @parcel/cache@2.0.1
└── lmdb-store@1.6.14
andrey@ru41 client %
To build the project, I actively use Parcel, which most likely depends on lmdb-store
and node-lmdb
Yes, lmdb-store is a dependency of parcel. lmdb-store@1.6.14 already includes a prebuild for linux-x64+node-v16 (you could verify it at node_modules/lmdb-store/prebuilds/linux-64/node-abi94.node). So the prebuild must somehow be failing to work on your linux system. Are you using a published dockerfile? I'd be glad to try to run a github action to see if I could reproduce this on the same docker container you are using.
(And if you don't mind, we could file/move this to lmdb-store, just for better organization)
Yes, lmdb-store is a dependency of parcel. lmdb-store@1.6.14 already includes a prebuild for linux-x64+node-v16 (you could verify it at node_modules/lmdb-store/prebuilds/linux-64/node-abi94.node). So the prebuild must somehow be failing to work on your linux system. Are you using a published dockerfile? I'd be glad to try to run a github action to see if I could reproduce this on the same docker container you are using.
(And if you don't mind, we could file/move this to lmdb-store, just for better organization)
Sure! We can move this issue to more appropriate place.
Can we automatically move this issue to lmdb-store?
Github gives a 500 error when I try to transfer the issue, so I guess that won't work right now.
Good. Maybe a little later it will turn out to be move this issue to lmdb-store automatically
What about your request to reproduce this error in the github actions: you probably need a docker file as well as all the dependencies from the package.json file from the project where this issue had occured right?
Well, I was just going to try to run the current lmdb-store test suite github action using your docker file.
Another thing that would be a helpful for debugging would be for you to try to run your build with npm i --ignore-scripts
. This should force it to use the prebuilds and not attempt any compilation, and should then eventually show/report if and what error is occurring in the prebuilds.
I threw out everything unnecessary from the package.json
and Dockerfile
files, left only what could theoretically cause an error, I hope 🙏 it will start in github action.
However I could run it on my local machine:
andrey@ru41 src % docker build -t test:v1 --no-cache -f Dockerfile .
[+] Building 53.1s (15/15) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 575B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/node:lts-alpine3.14 0.8s
=> [internal] load build context 15.8s
=> => transferring context: 496.82MB 15.7s
=> CACHED [ 1/10] FROM docker.io/library/node:lts-alpine3.14@sha256:60ef0bed1dc2ec835cfe3c4226d074fdfaba571fd6 0.0s
=> [ 2/10] RUN apk update && apk add util-linux 3.9s
=> [ 3/10] RUN apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python 5.2s
=> [ 4/10] RUN python3 -m ensurepip 6.5s
=> [ 5/10] RUN pip3 install --no-cache --upgrade pip setuptools 5.8s
=> [ 6/10] RUN apk add --update alpine-sdk 10.9s
=> [ 7/10] WORKDIR /src 0.0s
=> [ 8/10] COPY [/client/package*, ./] 0.2s
=> [ 9/10] COPY client/ ./ 5.0s
=> [10/10] RUN npm i --no-optional 7.7s
=> exporting to image 7.0s
=> => exporting layers 7.0s
=> => writing image sha256:18a33f956b95c5900164a0d5321c2b0a7b91fc797a61180f7841cd6e2d0fa708 0.0s
=> => naming to docker.io/library/test:v1 0.0s
andrey@ru41 src %
To reproduce this issue in gitlab/github actions try this:
Dockerfile:
FROM node:lts-alpine3.14 AS base
RUN apk update && apk add util-linux
# Install python/pip
ENV PYTHONUNBUFFERED=1
RUN apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python
RUN python3 -m ensurepip
RUN pip3 install --no-cache --upgrade pip setuptools
RUN apk add --update alpine-sdk
WORKDIR /
COPY ["./package.json", "./"]
RUN npm i --no-optional
ENTRYPOINT ["docker-entrypoint.sh"]
#CMD ["php-fpm"]
package.json:
{
"engines": {
"node": "~16",
"npm": "~8"
},
"private": true,
"license": "ISC",
"devDependencies": {
"@parcel/optimizer-cssnano": "^2.0.1",
"@parcel/packager-css": "^2.0.1",
"@parcel/packager-raw-url": "^2.0.1",
"@parcel/packager-xml": "^2.0.1",
"@parcel/transformer-image": "^2.0.1",
"@parcel/transformer-postcss": "^2.0.1",
"@parcel/transformer-pug": "^2.0.1",
"@parcel/transformer-sass": "^2.0.1",
"@parcel/transformer-vue": "^2.0.1",
"@parcel/transformer-webmanifest": "^2.0.1",
"@parcel/transformer-xml": "^2.0.1",
"@types/node": "^16.11.9",
"add": "^2.0.6",
"autoprefixer": "^10.4.0",
"babel-preset-env": "^1.7.0",
"babel-preset-react": "^6.24.1",
"css-loader": "^6.5.1",
"eslint": "^8.3.0",
"eslint-plugin-react": "^7.27.1",
"http-proxy-middleware": "^1.3.1",
"node-sass": "^6.0.1",
"parcel": "^2.0.1",
"parcel-reporter-static-files-copy": "^1.3.1",
"posthtml-hash": "^1.2.2",
"posthtml-modules": "^0.7.4",
"pug": "^3.0.2",
"src": "1.0.0",
"stylelint": "^14.1.0",
"stylelint-config-sass-guidelines": "^9.0.1",
"stylelint-scss": "^4.0.0",
"stylelint-selector-bem-pattern": "^2.1.1"
},
"dependencies": {
"@fortawesome/fontawesome-free": "^5.15.4",
"axios": "^0.24.0",
"bootstrap": "^4.3.1",
"classnames": "^2.2.6",
"core-js": "^3.9.0",
"current-script-polyfill": "^1.0.0",
"dayjs": "^1.10.7",
"details-polyfill": "^1.1.0",
"echarts": "^5.1.1",
"element-plus": "^1.2.0-beta.3",
"highcharts": "^9.1.1",
"highcharts-react-official": "^3.0.0",
"idempotent-babel-polyfill": "^7.4.4",
"inputmask": "^5.0.6",
"jquery": "^3.6.0",
"jquery.scrollbar": "^0.2.11",
"jsencrypt": "^3.1.0",
"lodash": "^4.17.15",
"maskedinput": "^3.3.11",
"object-fit-images": "^3.2.4",
"popper.js": "^1.16.0",
"qs": "^6.9.1",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"regenerator-runtime": "^0.13.7",
"reveal.js": "^4.1.3",
"services": "0.0.3",
"signalr": "^2.4.1",
"svgxuse": "^1.2.6",
"symbol-es6": "^0.1.2",
"uuid": "^8.3.2",
"vue": "^3.2.11",
"vue-hot-reload-api": "^2.3.4",
"vue-i18n": "^9.1.7",
"vue-router": "^3.1.5",
"webpack": "^5.64.2",
"zrender": "^5.2.1"
},
"optionalDependencies": {
"sass-migrator": "^1.2.5"
},
"bundleDependencies": [
"webpack"
]
}
So I ran this github action, which I think should be a close resemblance of your docker configuration, but specifically building and testing lmdb-store: https://github.com/DoctorEvidence/lmdb-js/runs/4301924530?check_suite_focus=true The build succeeded and lmdb-store ran, however, the test suite completely failed due to permission denied errors on any attempt to create the db files. I realize this isn't exactly the same error you were seeing, however, this is an error I have never seen before on any of the other dockers I have run/tested lmdb-store on, and it seems similar and possibly the same cause as your errors. I did some comparisons and it seems this permissions issue manifests specifically for the alpine node 16 docker images, but not for alpine node 12 or 14. I don't know if there was a change to these docker image that altered the permissions?
Can you try a couple things for me:
npm i --ignore-scripts
?Thank you for the help!
(And further attempts to do an npm install with your package.json deps on node:lts-alpine3.14 seemed to install without any errors, and without any compilation).
About things you are asking for:
file:
a link to a local package that only works from NPM 7 and up that comes with Node 15th and up ... When installing using YARN, the package installation logic falls on YARN and the error may not occur, but this is not certain.npm ci
command to npm i
and started the build process in the gitlab pipeline, but the error did not return 🤷♂️. The build completed successfully. Than I tryed to change in the Dockerfile npm i --no-optional
command to npm i --ignore-scripts
and again started the build process in the gitlab pipeline, but the build is still successfull. I could not throw the previous error again 🤷♂️.Step 20/43 : RUN npm i --ignore-scripts
---> Running in 1c51af6be1d7
npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
npm WARN deprecated @babel/polyfill@7.4.4: 🚨 This package has been deprecated in favor of separate inclusion of a polyfill and regenerator-runtime (when needed). See the @babel/polyfill docs (https://babeljs.io/docs/en/babel-polyfill) for more information.
npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
npm WARN deprecated core-js@2.6.12: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm WARN deprecated core-js@2.6.12: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
added 1149 packages, and audited 1152 packages in 36s
194 packages are looking for funding
run `npm fund` for details
19 moderate severity vulnerabilities
To address all issues (including breaking changes), run:
npm audit fix --force
Run `npm audit` for details.
Removing intermediate container 1c51af6be1d7
Ok, I also tried using npm ci
instead of npm i
in my github action, but still no luck in reproducing an error.
Yes, the error no longer occurs. Reason not found
Hello, after updating the node to the 16th version of the gitlab for dockerfile launched in the pipeline, I began to receive this error. Then I tried to install this package directly on my computer under control of macOS and the error was confirmed.
Can you please push a patch for this issue?
Pipeline install
Local install