nvm-sh / nvm

Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions
MIT License
79.8k stars 7.99k forks source link

curl-install-sh: `fatal: bad object` from git clone steps #3393

Open avengerx opened 3 months ago

avengerx commented 3 months ago

Operating system and version:

Windows Server 2012 R2 NodeJS 20.15.1 Git for Windows 2.45.2.windows.1 Cygwin1.dll 3.4.5

nvm debug output:

```sh # command not found? ```

nvm ls output:

```sh # command... not found. ```

How did you install nvm?

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

What steps did you perform?

Ran the install command

What happened?

install.sh script output ``` % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16555 100 16555 0 0 140k 0 --:--:-- --:--:-- --:--:-- 141k => Downloading nvm from git to '/home/avenger/.nvm' => Cloning into '/home/avenger/.nvm'... remote: Enumerating objects: 369, done. remote: Counting objects: 100% (369/369), done. remote: Compressing objects: 100% (315/315), done. remote: Total 369 (delta 42), reused 166 (delta 27), pack-reused 0 Receiving objects: 100% (369/369), 368.22 KiB | 30.68 MiB/s, done. Resolving deltas: 100% (42/42), done. remote: Enumerating objects: 366, done. remote: Counting objects: 100% (366/366), done. remote: Compressing objects: 100% (315/315), done. remote: Total 366 (delta 42), reused 162 (delta 25), pack-reused 0 Receiving objects: 100% (366/366), 365.90 KiB | 30.49 MiB/s, done. Resolving deltas: 100% (42/42), done. fatal: bad object cdc56687ada0c47f97632b0049b0446aadbe3a3a error: remote did not send all necessary objects Failed to fetch origin with v0.39.7. Please report this! ```

What did you expect to happen?

Install nvm.

Is there anything in any of your profile files that modifies the PATH?

~/.bash_profile

If you are having installation issues, or getting "N/A", what does curl -I --compressed -v https://nodejs.org/dist/ print out?

```sh $ curl -I --compressed -v https://nodejs.org/dist/ * STATE: INIT => CONNECT handle 0xa0001eef8; line 1909 (connection #-5000) * Added connection 0. The cache now contains 1 members * STATE: CONNECT => RESOLVING handle 0xa0001eef8; line 1955 (connection #0) * [CONN-0-0] setup, init filter chain * [CONN-0-0] cf_add(filter=SOCKET) * [CONN-0-0] cf_add(filter=SSL) * [CONN-0-0][CF-SOCKET] setup(remotehost=nodejs.org) * STATE: RESOLVING => CONNECTING handle 0xa0001eef8; line 2029 (connection #0) * family0 == v4, family1 == v6 * Trying 104.20.22.46:443... * [CONN-0-0] connect(block=0)-> 0, done=0 * Connected to nodejs.org (104.20.22.46) port 443 (#0) * ALPN: offers h2 * ALPN: offers http/1.1 * CAfile: /etc/pki/tls/certs/ca-bundle.crt * CApath: none * [CONN-0] Didn't find Session ID in cache for host HTTPS://nodejs.org:443 * [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS handshake, Client hello (1): * [CONN-0-0] connect(block=0)-> 0, done=0 * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Server hello (2): * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Certificate (11): * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, CERT verify (15): * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Finished (20): * [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN: server accepted h2 * Server certificate: * subject: CN=*.nodejs.org * start date: Feb 28 00:00:00 2024 GMT * expire date: Mar 30 23:59:59 2025 GMT * subjectAltName: host "nodejs.org" matched cert's "nodejs.org" * issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA * SSL certificate verify ok. * [CONN-0-0] connect(block=0)-> 0, done=1 * STATE: CONNECTING => PROTOCONNECT handle 0xa0001eef8; line 2073 (connection #0) * [CONN-0-0] connect(block=0)-> 0, done=1 * STATE: PROTOCONNECT => DO handle 0xa0001eef8; line 2103 (connection #0) * Using HTTP2, server supports multiplexing * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * h2h3 [:method: HEAD] * h2h3 [:path: /dist/] * h2h3 [:scheme: https] * h2h3 [:authority: nodejs.org] * h2h3 [user-agent: curl/7.87.0] * h2h3 [accept: */*] * h2h3 [accept-encoding: deflate, gzip, br, zstd] * Using Stream ID: 1 (easy handle 0xa0001eef8) > HEAD /dist/ HTTP/2 > Host: nodejs.org > user-agent: curl/7.87.0 > accept: */* > accept-encoding: deflate, gzip, br, zstd > * STATE: DO => DID handle 0xa0001eef8; line 2199 (connection #0) * multi changed, check CONNECT_PEND queue * STATE: DID => PERFORMING handle 0xa0001eef8; line 2318 (connection #0) * Curl_readwrite(handle=0xa0001eef8) -> 0 * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * [CONN-0] Didn't find Session ID in cache for host HTTPS://nodejs.org:443 * [CONN-0] Added Session ID to cache for HTTPS://nodejs.org:443 [server] * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * [CONN-0] Found Session ID in cache for host HTTPS://nodejs.org:443 * old SSL session ID is stale, removing * [CONN-0] Added Session ID to cache for HTTPS://nodejs.org:443 [server] * readwrite_data(handle=0xa0001eef8) -> 0 * Curl_readwrite(handle=0xa0001eef8) -> 0 * readwrite_data(handle=0xa0001eef8) -> 0 * Curl_readwrite(handle=0xa0001eef8) -> 0 * HTTP/2 found, allow multiplexing < HTTP/2 200 HTTP/2 200 < date: Tue, 23 Jul 2024 22:31:52 GMT date: Tue, 23 Jul 2024 22:31:52 GMT < content-type: text/html content-type: text/html < cache-control: public, max-age=3600, s-maxage=14400 cache-control: public, max-age=3600, s-maxage=14400 < last-modified: Tue, 23 Jul 2024 19:35:45 GMT last-modified: Tue, 23 Jul 2024 19:35:45 GMT < cf-cache-status: HIT cf-cache-status: HIT < age: 10355 age: 10355 < vary: Accept-Encoding vary: Accept-Encoding < strict-transport-security: max-age=31536000; includeSubDomains; preload strict-transport-security: max-age=31536000; includeSubDomains; preload < x-content-type-options: nosniff x-content-type-options: nosniff < server: cloudflare server: cloudflare < cf-ray: 8a7f1f8aab5c5794-IAD cf-ray: 8a7f1f8aab5c5794-IAD < content-encoding: br content-encoding: br < * readwrite_data(handle=0xa0001eef8) -> 0 * Curl_readwrite(handle=0xa0001eef8) -> 0 * STATE: PERFORMING => DONE handle 0xa0001eef8; line 2517 (connection #0) * multi_done: status: 0 prem: 0 done: 0 * Connection #0 to host nodejs.org left intact * Expire cleared (transfer 0xa0001eef8) ```

bottomline

If I download/checkout and edit install.sh and change all --git-dir="${INSTALL_DIR}/.git" --work-tree="${INSTALL_DIR}" with just -C "${INSTALL_DIR}" the script runs to completion.

`$ cat install.sh | bash` ``` => nvm is already installed in /home/user/.nvm, trying to update using git => * (HEAD detached at FETCH_HEAD) master => Compressing and cleaning up git repository => Appending nvm source string to /home/user/.bashrc => Appending bash_completion source string to /home/user/.bashrc => You currently have modules installed globally with `npm`. These will no => longer be linked to the active version of Node when you install a new node => with `nvm`; and they may (depending on how you construct your `$PATH`) => override the binaries of modules installed with `nvm`: C:\Users\user\AppData\Roaming\npm └── nvm@0.0.4 => If you wish to uninstall them at a later point (or re-install them under your => `nvm` Nodes), you can remove them from the system Node as follows: $ nvm use system $ npm uninstall -g a_module => Close and reopen your terminal to start using nvm or run the following to use it now: export NVM_DIR="$HOME/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion ``` This happens because I'm running git for windows from a cygwin shell, so the linux-like path provided in `--git-dir` doesn't get translated correctly. It seems both `-C` and `--work-tree` are translated correctly. I didn't retry the script just dropping `--git-dir` as it would probably fail as my current working directory was not `~/.nvm`. So, perhaps replacing the command to use `-C` could be a bit more portable. Maybe checking `"${OSTYPE}" == "cygwin"` could create an exception, if using both work-tree and git-dir is meant to solve issues with a specific OS/environment. Hope this helps improve NVM! *edit:* this issue is probably the same reported in #3329.
ljharb commented 3 months ago

The nvm@0.0.4 output indicates you have an npm-installed nvm, which isn't actually a working version of anything.

The issue with the git syntax suggests your git version is unsupported - we require v1.7.10, per the readme. What version do you have?

avengerx commented 3 months ago

I had installed the nvm@0.0.4 where it informed me I needed to install it from the shell script. I understand it didn't do anything, I just didn't bother to remove it back then.

Git is 2.45.2 as I stated in the top (operating system/version).

I don't know, to me it really look like just a cygwin-to-git4win path conversion issue. Again, it installed when I switched to -C (which should be equivalent to the pair of git arguments). Thing is, I am calling git for windows with linux(cygwin) paths, thus the paths (in /home/user/.nvm format) doesn't work, as it would expect something like c:\cygwin64\home\user\.nvm, which cygwin can translate when -C is passed.

Unless I'm mistaken, cygwin automatically translates paths passed to non-cygwin commands. When it's in the format -C /home/user/.nvm. But it doesn't really try it when the arguments are tied --git-dir="/home/user/.nvm".

This issue should only trigger when using WSL or cygwin with non-native builds of git. The solution that works for me though, probably doesn't for WSL.

ljharb commented 3 months ago

What happens if you use a cygwin-installed git, so that it's not using the windows git?

avengerx commented 3 months ago

Using cygwin-installed git (yet, of course, Windows-installed node), it works. Tested in a somewhat similar station with git 2.31.1, node 14.17.6, and cygwin1.dll 3.2.0.

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
``` % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16555 100 16555 0 0 22252 0 --:--:-- --:--:-- --:--:-- 22251 => Downloading nvm from git to '/home/avenger/.nvm' => Cloning into '/home/avenger/.nvm'... remote: Enumerating objects: 369, done. remote: Counting objects: 100% (369/369), done. remote: Compressing objects: 100% (315/315), done. remote: Total 369 (delta 42), reused 166 (delta 27), pack-reused 0 Receiving objects: 100% (369/369), 368.22 KiB | 611.00 KiB/s, done. Resolving deltas: 100% (42/42), done. * (HEAD detached at FETCH_HEAD) master => Compressing and cleaning up git repository => Appending nvm source string to /home/avenger/.bashrc => Appending bash_completion source string to /home/avenger/.bashrc => You currently have modules installed globally with `npm`. These will no => longer be linked to the active version of Node when you install a new node => with `nvm`; and they may (depending on how you construct your `$PATH`) => override the binaries of modules installed with `nvm`: C:\Users\avenger\AppData\Roaming\npm +-- heroku@7.67.1 +-- json2yaml@1.1.0 +-- proggy@0.1.0 +-- retypeapp-win-x64@2.3.0-202205021640 +-- typescript@3.8.3 +-- windows-build-tools@5.2.2 `-- yarn@1.22.11 => If you wish to uninstall them at a later point (or re-install them under your => `nvm` Nodes), you can remove them from the system Node as follows: $ nvm use system $ npm uninstall -g a_module => Close and reopen your terminal to start using nvm or run the following to use it now: export NVM_DIR="$HOME/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion ```
ljharb commented 3 months ago

Windows-installed node shouldn't matter because nvm doesn't itself use node.

OK, thanks! That means that nvm's install script either needs to detect windows-installed git, ideally with feature detection, and either a) error out, or b) change the git commands so they work.

Alternatively, cygwin could (and probably should) add support for --git-dir if it already supports -C?

avengerx commented 3 months ago

Windows-installed node shouldn't matter because nvm doesn't itself use node.

Oh, sure, right!

OK, thanks! That means that nvm's install script either needs to detect windows-installed git, ideally with feature detection, and either a) error out, or b) change the git commands so they work.

Ya, erroring (a) could be frustrating to the user, but not bulldozing directories where they shouldn't will definitively help at least measure the worth of treating this edge case. Adapting the commands (b) should involve some magic I'm not sure is worth for the script maintainability over the long run. It would involve, after resolving ${INSTALL_DIR} something like this

conceptual script ```bash if [ "${OSTYPE}" == "cygwin" ]; then if ldd "$(which git)" | egrep -q "^[ "$'\t'"]*cygwin1.dll "; then INSTALL_DIR_CONVERTED="$(cygpath -w "$(readlink -f "${INSTALL_DIR}")")" fi fi ``` Basically, if under cygwin -and- the `git` command in path does not link against `cygwin1.dll`, we can safely tell it's windows version. Then path conversion _only for git calls_ should take place.
Git-windows ldd output ``` $ ldd "$(which git)" ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x7ff81c780000) KERNEL32.DLL => /cygdrive/c/Windows/system32/KERNEL32.DLL (0x7ff81b6d0000) KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll (0x7ff819cf0000) msvcrt.dll => /cygdrive/c/Windows/system32/msvcrt.dll (0x7ff81ba00000) SHELL32.dll => /cygdrive/c/Windows/system32/SHELL32.dll (0x7ff819ed0000) SHLWAPI.dll => /cygdrive/c/Windows/system32/SHLWAPI.dll (0x7ff819e70000) USER32.dll => /cygdrive/c/Windows/system32/USER32.dll (0x7ff81c220000) combase.dll => /cygdrive/c/Windows/SYSTEM32/combase.dll (0x7ff81bff0000) GDI32.dll => /cygdrive/c/Windows/system32/GDI32.dll (0x7ff81bea0000) RPCRT4.dll => /cygdrive/c/Windows/system32/RPCRT4.dll (0x7ff81b590000) SspiCli.dll => /cygdrive/c/Windows/system32/SspiCli.dll (0x7ff819980000) sechost.dll => /cygdrive/c/Windows/SYSTEM32/sechost.dll (0x7ff81bb60000) ```
Git-cygwin ldd output ``` $ ldd $(which git) ntdll.dll => /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffc42e30000) KERNEL32.DLL => /cygdrive/c/WINDOWS/System32/KERNEL32.DLL (0x7ffc42cd0000) KERNELBASE.dll => /cygdrive/c/WINDOWS/System32/KERNELBASE.dll (0x7ffc40760000) cygintl-8.dll => /usr/bin/cygintl-8.dll (0x3fd310000) cygz.dll => /usr/bin/cygz.dll (0x3fa050000) cygiconv-2.dll => /usr/bin/cygiconv-2.dll (0x3fcdb0000) cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000) ```

So, not sure this edgy case is worth this much trouble.

Alternatively, cygwin could (and probably should) add support for --git-dir if it already supports -C?

This brings us to another long piece. In summary, I was mistaken. Cygwin doesn't do any path conversion and it worked by luck. The -C was required to get rid of the bad object error, though, which I'm still not completely sure why it happened, as all other git steps were "going through". Details follows.

Gory details

I was double-checking this... and seems cygwin doesn't really do any path translation. The output of [ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm" is /home/user/.nvm

And I tried some commands to see what happens. And turns out that git-for-windows version does interpret the path and creates the directories.

Say, I have /home/user/ as my home directory, which from cygwin would translate to C:\cygwin64\home\user\.

I noticed what happened here was, all stuff was installed and set up in C:\home\user\.nvm. When I tried git init /home/user/.nvm (based on v0.39.7's install.sh), what I got was:

Initialized empty Git repository in C:/home/user/.nvm/.git/

Thus, I see there's no conversion made by cygwin at all, git-for-win just install everything outside of the actual user's home. Just, for some reason, --git-dir= and/or --work-tree= mostly works. It clones in that "foreign" directory, checks stuff out. All having the "build environment" actually outside actual user home. But then that step to check out the tag fails. Even though it shouldn't by looking into C:\home\user\.nvm (which is not the same ascygwin's /home/user/.nvm)

Then I believe -C does work because it is equivalent to the syntax used for git clone, that just passes the install path (that means, "change to that directory and do the commands").

In the end, I had /home/user/.nvm (install.sh:152)[https://github.com/nvm-sh/nvm/blob/v0.39.7/install.sh#L152] but also had C:\home\user\.nvm (which corresponds to /cygdrive/c/home/user/.nvm in cygwin).

It indeed worked because, at least (a) I manually cloned nvm-sh/nvm into real cygwin /home/user/.nvm and checked out tag (b) The commands from the install script succeeded with -C /home/user/.nvm being interpreted as -C C:\home\user\.nvm

So I had very close branches in both directories, so whenever the script called git-win, it worked on the "foreign" repo, but whenever the rest of the script referred to the actual directory, it found what it expected. 😄

Then again, no telling what else was left not done in one directory but done in another, if you follow the thought. It seemed to have worked, but I didn't try too many things, I'm new with nvm.

bottomline

In the end, for my use case, I was setting up a server service, something I'd need to run as network service, so as nvm is something user-scoped it wouldn't really help and I will need to keep node up-to-date in the server the oldschool way.

So that's another reason it could be safer to just error out and see if more people raise here hitting this wall.

This issue might also be affected by git-bash-here (git for windows context menu item). I believe they use MSYS, and it seems this one does auto path conversion so...