sameersbn / docker-gitlab

Dockerized GitLab
http://www.damagehead.com/docker-gitlab/
MIT License
7.91k stars 2.14k forks source link

node modules not updated #2964

Closed th-2021 closed 4 months ago

th-2021 commented 4 months ago

During assets compile I get error that some node modules are missing. This is with all newer versions e.g. 17.0.3

kkimurak commented 4 months ago

@th-2021 Could you provide more detail? Do you mean following error on using relative url enabled (that is said that can be safely ignored)?

docker log of `sameersbn/gitlab:17.0.3`, setting `GITLAB_RELATIVE_URL=/gitlab` ```log warning Error running install script for optional dependency: "/home/git/gitlab/node_modules/tree-sitter-json: Command failed. Exit code: 1 Command: node-gyp rebuild Arguments: Directory: /home/git/gitlab/node_modules/tree-sitter-json Output: gyp info it worked if it ends with ok gyp info using node-gyp@10.1.0 gyp info using node@20.15.0 | linux | x64 gyp info find Python using Python version 3.8.10 found at \"/usr/bin/python3\" gyp http GET https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz gyp http 200 https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz gyp http GET https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt gyp http 200 https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt gyp info spawn /usr/bin/python3 gyp info spawn args [ gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/home/git/gitlab/node_modules/tree-sitter-json/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/home/git/.cache/node-gyp/20.15.0/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/home/git/.cache/node-gyp/20.15.0', gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/home/git/.cache/node-gyp/20.15.0/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/home/git/gitlab/node_modules/tree-sitter-json', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] gyp ERR! build error gyp ERR! stack Error: not found: make gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:16:17) gyp ERR! stack at which (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:77:9) gyp ERR! stack at async doWhich (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:112:22) gyp ERR! stack at async loadConfigGypi (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:77:7) gyp ERR! stack at async build (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:35:3) gyp ERR! stack at async run (/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js:81:18) gyp ERR! System Linux 3.10.0-1160.119.1.el7.x86_64 gyp ERR! command \"/usr/bin/node\" \"/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\" gyp ERR! cwd /home/git/gitlab/node_modules/tree-sitter-json gyp ERR! node -v v20.15.0 gyp ERR! node-gyp -v v10.1.0 gyp ERR! not ok" info This module is OPTIONAL, you can safely ignore this error info This module is OPTIONAL, you can safely ignore this error warning Error running install script for optional dependency: "/home/git/gitlab/node_modules/tree-sitter-yaml: Command failed. Exit code: 1 Command: node-gyp rebuild Arguments: Directory: /home/git/gitlab/node_modules/tree-sitter-yaml Output: gyp info it worked if it ends with ok gyp info using node-gyp@10.1.0 gyp info using node@20.15.0 | linux | x64 gyp info find Python using Python version 3.8.10 found at \"/usr/bin/python3\" gyp http GET https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz gyp http 200 https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz gyp http GET https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt gyp http 200 https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt gyp info spawn /usr/bin/python3 gyp info spawn args [ gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/home/git/gitlab/node_modules/tree-sitter-yaml/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/home/git/.cache/node-gyp/20.15.0/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/home/git/.cache/node-gyp/20.15.0', gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/home/git/.cache/node-gyp/20.15.0/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/home/git/gitlab/node_modules/tree-sitter-yaml', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] gyp ERR! build error gyp ERR! stack Error: not found: make gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:16:17) gyp ERR! stack at which (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:77:9) gyp ERR! stack at async doWhich (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:112:22) gyp ERR! stack at async loadConfigGypi (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:77:7) gyp ERR! stack at async build (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:35:3) gyp ERR! stack at async run (/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js:81:18) gyp ERR! System Linux 3.10.0-1160.119.1.el7.x86_64 gyp ERR! command \"/usr/bin/node\" \"/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\" gyp ERR! cwd /home/git/gitlab/node_modules/tree-sitter-yaml gyp ERR! node -v v20.15.0 gyp ERR! node-gyp -v v10.1.0 gyp ERR! not ok" warning Error running install script for optional dependency: "/home/git/gitlab/node_modules/tree-sitter: Command failed. Exit code: 1 Command: prebuild-install || node-gyp rebuild Arguments: Directory: /home/git/gitlab/node_modules/tree-sitter Output: prebuild-install warn install No prebuilt binaries found (target=20.15.0 runtime=node arch=x64 libc= platform=linux) gyp info it worked if it ends with ok gyp info using node-gyp@10.1.0 gyp info using node@20.15.0 | linux | x64 gyp info find Python using Python version 3.8.10 found at \"/usr/bin/python3\" gyp http GET https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz gyp http 200 https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz gyp http GET https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt gyp http 200 https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt gyp info spawn /usr/bin/python3 gyp info spawn args [ gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/home/git/gitlab/node_modules/tree-sitter/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/home/git/.cache/node-gyp/20.15.0/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/home/git/.cache/node-gyp/20.15.0', gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/home/git/.cache/node-gyp/20.15.0/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/home/git/gitlab/node_modules/tree-sitter', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] gyp ERR! build error gyp ERR! stack Error: not found: make gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:16:17) gyp ERR! stack at which (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:77:9) gyp ERR! stack at async doWhich (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:112:22) gyp ERR! stack at async loadConfigGypi (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:77:7) gyp ERR! stack at async build (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:35:3) gyp ERR! stack at async run (/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js:81:18) gyp ERR! System Linux 3.10.0-1160.119.1.el7.x86_64 gyp ERR! command \"/usr/bin/node\" \"/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\" gyp ERR! cwd /home/git/gitlab/node_modules/tree-sitter gyp ERR! node -v v20.15.0 gyp ERR! node-gyp -v v10.1.0 gyp ERR! not ok" info This module is OPTIONAL, you can safely ignore this error ```
th-2021 commented 4 months ago

With relative url set. When I run the image I get make and g++ not found.

With make and g++ added I get:

kkimurak commented 4 months ago

Well, gcc and make are listed in BUILD_DEPENDENCIES in build script. They are purged at the end of the script.

We have some choise:

The second idea would be acceptable.
When I was working on preparing postgresl-13 (#2685), the initial idea for that pull request was to install the appropriate version of postgresql-client at runtime. After receiving a review from the maintainer @sachilles saying that he wanted to avoid requiring an internet connection, I rewrote the patch to pre-install a possible version of postgresql-client in the image. It has been merged.

This may be pointless since we (relative url users) install the necessary node modules at runtime anyway, but I'll write this as advice anyway. I means that I have no plans to create a pull request on this subject at this time.

th-2021 commented 4 months ago

I tried the upgrades again and the error can be ignored. With one version I had broken assets and thought that this is the issue, but it must have been something different. It worked now.

shplishka commented 4 months ago

Is there any version where this doesn't happen?

kkimurak commented 4 months ago

@shplishka No. More precisely, the root cause of this problem (a situation where gcc and make are required at runtime) is not known. According to @th-2021 's report, it may be caused by corrupted assets but not clear.

I've been running gitlab with the relative url feature enabled in all versions since at least 13.10.3 until 17.2.0 (latest version available today), and no such situation occurred on my environment.

If you are facing similar issue, providing your detailed setup may help someone who interested in this topic to debug.

th-2021 commented 4 months ago

I think my issue with broken assets was unrelated to the node modulesAm 24.07.2024 11:10 schrieb Kazunori Kimura @.***>: @shplishka No. More precisely, the root cause of this problem (a situation where gcc and make are required at runtime) is not known. According to @th-2021 's report, it may be caused by corrupted assets but not clear. I've been running gitlab with the relative url feature enabled in all versions since at least 13.10.3 until 17.2.0 (latest version available today), and no such situation occurred on my environment. If you are facing similar issue, providing your detailed setup may help someone who interested in this topic to debug.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

shplishka commented 4 months ago

Hi @th-2021 and @kkimurak,

Thank you for your previous assistance. I am facing another problem related to this behavior. When I use a relative URL in an air-gapped cluster that doesn't have access to npm, Artifactory, etc., the image crashes when it tries to install the node modules.

I'm not sure how to resolve this issue. Any advice would be appreciated.

kkimurak commented 4 months ago

@shplishka
Unfortunately, the relative URL feature should not work without an active internet connection - as this image, sameersbn/gitlab, is published with node_modules removed at the end of build to reduce image size.

Anyway, could you create a new issue about it (relative url does not work if there is no access to npm registry and so on) ? I think it's not a good idea to continue discussing on an issue that has already been closed, and I think it's different from what was originally reported for this issue.

Submitting a merge request is much better. You should just modify the line.

diff --git a/assets/build/install.sh b/assets/build/install.sh
index 47fc4620..fb330b23 100755
--- a/assets/build/install.sh
+++ b/assets/build/install.sh
@@ -484,5 +484,5 @@ rm -rf /var/lib/apt/lists/*

 # clean up caches
 rm -rf ${GITLAB_HOME}/.cache ${GITLAB_HOME}/.bundle ${GITLAB_HOME}/go
-rm -rf /root/.cache /root/.bundle ${GITLAB_HOME}/gitlab/node_modules
+rm -rf /root/.cache /root/.bundle
 rm -r /tmp/*

I can report how much the image size changes.

In any case, the maintainer will make the final decision. You can mention to active maintainer (@sachilles) on demand.