search

volta-cli / volta

Volta: JS Toolchains as Code. ⚡
https://volta.sh
Other
10.95k stars 227 forks source link

Could not download node #1523

Open xiaoLangtou opened 1 year ago

xiaoLangtou commented 1 year ago

When using the commands volta install node@版本号 and volta pia node@版本号 to download Node.js, you encountered the following error message, but network issues were not detected after checking. You can also download the corresponding Node.js file from the official Node.js website.

➜ volta install node@18.10.0
error: Could not download node@18.10.0
from https://nodejs.org/dist/v18.10.0/node-v18.10.0-darwin-arm64.tar.gz
felipecrs commented 1 year ago

I'm having a similar error, happening intermittently in my CI/CD pipelines since a few days:

[2023-06-30T14:00:50.674Z] #28 0.443 + volta install node@18 npm@9 markdown-magic@1 semver@7 ts-node@10

[2023-06-30T14:00:50.674Z] #28 575.7 error: Could not unpack Node v18.16.1

[2023-06-30T14:00:50.674Z] #28 575.7 

[2023-06-30T14:00:50.674Z] #28 575.7 Please ensure the correct version is specified.

[2023-06-30T14:00:50.674Z] #28 575.7 Error details written to /home/vscode/.volta/log/volta-error-2023-06-30_14_00_48.738.log

Another one failed with Node 16.

xiaoLangtou commented 1 year ago

I think you can try to solve this problem by setting up the mirror source of nodejs through hook, For example Create %LOCALAPPDATA%\Volta\hooks.json(windows) / ~/.volta/hooks.json(Linux/MacOS) Paste the following content:

{
    "node": {
        "index": {
            "template": "https://mirrors.tuna.tsinghua.edu.cn/nodejs-release/index.json"
        },
        "distro": {
            "template": "https://mirrors.tuna.tsinghua.edu.cn/nodejs-release/v{{version}}/node-v{{version}}-{{os}}-{{arch}}.tar.gz"
        }
    }
}
Jungho-Cheon commented 1 year ago

same here

> volta install node@v16

error: Could not download Node version registry
from https://nodejs.org/dist/index.json

Please verify your internet connection.
felipecrs commented 1 year ago

I think it would be nice if Volta itself could ship with some pre-defined mirrors for redundancy.

chriskrycho commented 1 year ago

Hi all, thanks for the report. I've occasionally seen the same in CI over the past few months. Unfortunately, this stuff is all outside our ability to do anything about: we just ask the hosts for the downloads, and if they have a network issue… they have a network issue. 🤷🏻‍♂️ Shipping with built-in/pre-configured mirrors is an interesting idea, but definitely out of scope for us at present, and as you all may have inferred from the delay in our responding, we're all very much buried with other things right now! That said, if someone wants to spearhead writing an RFC with design for how that would work (vs. for users to define mirrors to fall back to in scenarios like this using the existing hooks design), we'd welcome it at least as a good point for discussion.

yairEO commented 1 year ago 
> volta install node@18
error: Could not download node@18.17.1
from https://nodejs.org/dist/v18.17.1/node-v18.17.1-win-x64.zip

Please verify your internet connection and ensure the correct version is specified.

I can download the zip file from the URL just fine, in the browser, but Volta claims there are network issues.

felipecrs commented 1 year ago

Since this really became a pain point in my CI/CD pipelines, I wanted to share my temporary solution to it:

  1. I installed retry to all my docker containers which uses Volta:
  2. I changed my volta install node instructions to use it:
retry --tries 3 --sleep 15s -- volta install node@18

The good thing is that, even if this issue gets fixed anywhere, retry will not cause any harm since it exits as soon as it passes. Also, I figured I could use the same tool for many other scenarios in my containers (and it's just so small), that I'd rather keep it there.

dmt0 commented 1 year ago

I'm connecting over a VPN. That doesn't seem to matter for the browser, I can download Node just fine. But for Volta I had to disable VPN.

trydofor commented 1 year ago 
volta --verbose install node
[verbose] No custom hooks found
[verbose] Found valid cache of Node version index
[verbose] Found newest LTS node version (18.17.1) from https://nodejs.org/dist/index.json
[verbose] Acquiring lock on Volta directory: ~/.volta/volta.lock
[verbose] Downloading node@18.17.1 from https://nodejs.org/dist/v18.17.1/node-v18.17.1-darwin-x64.tar.gz
[verbose] Unpacking node into '~/.volta/tmp/.tmpu1NqMB'
[verbose] Unlocking Volta Directory

#### without proxy 
error: Could not unpack Node v18.17.1

Please ensure the correct version is specified.
[verbose] Error cause: failed to unpack `~/.volta/tmp/.tmpu1NqMB/node-v18.17.1-darwin-x64/bin/node`

Error cause: failed to unpack `~/.volta/tmp/.tmpu1NqMB/node-v18.17.1-darwin-x64/bin/node`

Error cause: failed to unpack `node-v18.17.1-darwin-x64/bin/node` into `~/.volta/tmp/.tmpu1NqMB/node-v18.17.1-darwin-x64/bin/node`

Error cause: Resource temporarily unavailable (os error 35)
Error details written to ~/.volta/log/volta-error-2023-09-18_16_11_34.765.log

#### with https_proxy

error: Could not unpack Node v18.17.1

Please ensure the correct version is specified.
[verbose] Error cause: failed to unpack `~/.volta/tmp/.tmpu3ZeJG/node-v18.17.1-darwin-x64/include/node/openssl/archs/darwin-i386-cc/asm/include/openssl/asn1.h`

Error cause: failed to unpack `~/.volta/tmp/.tmpu3ZeJG/node-v18.17.1-darwin-x64/include/node/openssl/archs/darwin-i386-cc/asm/include/openssl/asn1.h`

Error cause: failed to unpack `node-v18.17.1-darwin-x64/include/node/openssl/archs/darwin-i386-cc/asm/include/openssl/asn1.h` into `~/.volta/tmp/.tmpu3ZeJG/node-v18.17.1-darwin-x64/include/node/openssl/archs/darwin-i386-cc/asm/include/openssl/asn1.h`

Error cause: Resource temporarily unavailable (os error 35)
Error details written to ~/.volta/log/volta-error-2023-09-18_16_43_24.165.log
chriskrycho commented 1 year ago

Reading through this and related issues (e.g. #1546) the repeated comments about this showing up most commonly on CI and VPNs, combined with the detailed error reporting here are pretty suggestive that this may be related to a rate-limiting behavior on the host servers. Anyone using any of the hosted runners for GH Actions, for example, will be asking from the same address; and anyone pulling from inside a VPN likewise. (And direct access to the tarball, esp. in a browser, does not always run over the same VPN paths that access via curl etc. do! Though if you can confirm that it is, that'd be very interesting.) The net of that would be exactly the behavior seen here. We're discussing how we might fix that, including by improving the way the GH Action works, and documenting ways to avoid that when coming from inside a VPN.

Sorry this has been a hassle! It's frustrating to us, too. 😞

lisonge commented 12 months ago

This error is very frequent

https://github.com/gkd-kit/subscription/actions/runs/6274100781/job/17038860167

https://github.com/gkd-kit/subscription/actions/runs/6262568177/job/17005034345

https://github.com/gkd-kit/subscription/actions/runs/6262539779/job/17004936137

https://github.com/gkd-kit/subscription/actions/runs/6262522729/job/17004877700

https://github.com/gkd-kit/subscription/actions/runs/6262485089/job/17004746507

https://github.com/gkd-kit/subscription/actions/runs/6249994343/job/17002563652

Here's my solution,

      - uses: actions/setup-node@v3
        with:
          node-version: 20

Now I only use volta on my personal computer

felipecrs commented 12 months ago

How odd:

[2023-09-25T13:43:54.645Z] #22 [devcontainer 2/7] RUN retry --tries 3 --sleep 15s -- volta install node@18 npm@9 markdown-magic@1 semver@7 ts-node@10
[2023-09-25T13:46:31.131Z] #22 193.5 error: Could not unpack Node v18.18.0
[2023-09-25T13:46:31.131Z] #22 193.5 
[2023-09-25T13:46:31.131Z] #22 193.5 Please ensure the correct version is specified.
[2023-09-25T13:46:31.131Z] #22 193.5 Error details written to /home/vscode/.volta/log/volta-error-2023-09-25_13_46_30.309.log
[2023-09-25T13:46:31.131Z] #22 193.6 Before retry #1: sleeping 15s seconds
[2023-09-25T13:50:22.668Z] #22 424.9 error: Could not unpack Node v18.18.0
[2023-09-25T13:50:22.668Z] #22 424.9 
[2023-09-25T13:50:22.668Z] #22 424.9 Please ensure the correct version is specified.
[2023-09-25T13:50:22.668Z] #22 424.9 Error details written to /home/vscode/.volta/log/volta-error-2023-09-25_13_50_21.732.log
[2023-09-25T13:50:22.668Z] #22 425.0 Before retry #2: sleeping 15s seconds
[2023-09-25T13:51:59.117Z] #22 517.1 error: Could not unpack Node v18.18.0
[2023-09-25T13:51:59.117Z] #22 517.1 
[2023-09-25T13:51:59.117Z] #22 517.1 Please ensure the correct version is specified.
[2023-09-25T13:51:59.117Z] #22 517.1 Error details written to /home/vscode/.volta/log/volta-error-2023-09-25_13_51_53.891.log
[2023-09-25T13:51:59.117Z] #22 517.2 Before retry #3: sleeping 15s seconds
[2023-09-25T13:53:35.571Z] #22 617.4 error: Could not unpack Node v18.18.0
[2023-09-25T13:53:35.571Z] #22 617.4 
[2023-09-25T13:53:35.571Z] #22 617.4 Please ensure the correct version is specified.
[2023-09-25T13:53:35.571Z] #22 617.4 Error details written to /home/vscode/.volta/log/volta-error-2023-09-25_13_53_34.237.log
[2023-09-25T13:53:35.571Z] #22 617.5 Retries exhausted

And yes, this runs within my internal company's network, which is probably under a single external IP and multiple people could be running it at the same time.

artuska commented 11 months ago

I run into the same problem. Also, it does not matter is VPN on or off.


PS C:\Windows\system32> volta install node@16.20.2
error: Could not download node@16.20.2
from https://nodejs.org/dist/v16.20.2/node-v16.20.2-win-x64.zip

Please verify your internet connection and ensure the correct version is specified.```
lasergoat commented 11 months ago

At work, we run into this issue continuously. We have Dockerfiles which use Volta within the build steps, like this: 

FROM debian:bookworm-slim as volta
ARG VOLTA_VERSION=1.1.0
ENV BASH_ENV=~/.bashrc
ENV VOLTA_HOME=/opt/volta/.volta
ENV PATH=$VOLTA_HOME/bin:$PATH

WORKDIR /usr/src/app
COPY package*.json ./

RUN apt update \
  && apt install -y \
  ca-certificates \
  curl \
  --no-install-recommends
SHELL ["/bin/bash", "-c"]
RUN curl https://get.volta.sh | bash -s -- --version ${VOLTA_VERSION}
RUN volta install node

Frequently, our CICD (in both github actions and AWS codebuild) get these two errors:

It seems intermittent over time, if it's happening, all builds fail. I'm not sure the difference between the two errors, my assumption is "could not unpack" happens after a download has started and the network caused a partial download; whereas the "could not download" happens if the initial request to download fails right away.

Either way, my first thought was to cache the download in the container ahead of time and keep it in ECR; but now I'm leaning towards just removing Volta from the Dockerfile altogether and going back to using like FROM node:18....

Open to ideas here.

ympadilha commented 11 months ago

For the record, I'm running into the same issue at work, multiple different projects that use volta are suffering from intermittent failures in CI/CD

09:36:46  yarn
09:36:46  Volta error: Could not download node@16.17.1
09:36:46  from https://nodejs.org/dist/v16.17.1/node-v16.17.1-linux-x64.tar.gz
09:36:46  
09:36:46  Please verify your internet connection and ensure the correct version is specified.
09:36:46  Volta error: Error cause: HTTP failure (500 Internal Server Error)
vanstinator commented 11 months ago

Could volta provide a way for the user to override the repository where nodejs is fetched from, so that we could point it to something like an internal artifactory instance where we're caching the binary ourselves?

felipecrs commented 11 months ago

@vanstinator isn't this what https://github.com/volta-cli/volta/issues/1523#issuecomment-1615570013 is for?

vanstinator commented 11 months ago

@vanstinator isn't this what https://github.com/volta-cli/volta/issues/1523#issuecomment-1615570013 is for?

Yes 🤦🏻 😅

Thank you!

lgiraudel commented 11 months ago

I used this for the ~/.volta/hooks.json and it seems to work too:

{
  "node": {
    "index": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/index.json"
    },
    "distro": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/v{{version}}/node-v{{version}}-{{os}}-{{arch}}.tar.gz"
    }
  }
}
Cohors1316 commented 11 months ago

I used this for the ~/.volta/hooks.json and it seems to work too:

{
  "node": {
    "index": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/index.json"
    },
    "distro": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/v{{version}}/node-v{{version}}-{{os}}-{{arch}}.tar.gz"
    }
  }
}

This partially fixed my issue. Windows, no vpn, completely unable to download node with volta. It hung for a few minutes with no output before finally returning an error that says it could not download node, not really anything helpful in the log.

With this it at least started fetching node, then failed out with Could not download node@18.18.2 and now I just get an immediate could not download.

addendum: After a complete uninstall, reinstall issue persisted, adding the hook gave the same result. I then deleted the hook and volta installed node exactly as expected.

jeanyves-yang commented 10 months ago

I used this for the ~/.volta/hooks.json and it seems to work too:

{
  "node": {
    "index": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/index.json"
    },
    "distro": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/v{{version}}/node-v{{version}}-{{os}}-{{arch}}.tar.gz"
    }
  }
}

This partially fixed my issue. Windows, no vpn, completely unable to download node with volta. It hung for a few minutes with no output before finally returning an error that says it could not download node, not really anything helpful in the log.

With this it at least started fetching node, then failed out with Could not download node@18.18.2 and now I just get an immediate could not download.

addendum: After a complete uninstall, reinstall issue persisted, adding the hook gave the same result. I then deleted the hook and volta installed node exactly as expected.

So ... basically it seems that it is still flaky and the hooks / pointing to a specific mirror does not change anything?

jorismak commented 10 months ago

At the moment the Nodejs servers seem to have issues again (and all week apparently).

Volta queries them for downloads, but also to determine the latest versions. This hooks.json seems to not do anything with the "latest" param, which can make them still stall there I'm guessing.

I still find it weird that I can open my browser to nodejs.org and go to the download section and have very fast downloads, but volta's install (or fetch) command stalls and does nothing, or is very slow, and aborts halfway through, etc...

Volta uses the .tar.gz files while the download site lists the .tar.xz files, that might be a thing.

But again: If volta has issues with nodejs-server issues, why is my browser working fine on that same site. Is it their load-balancing / redirecting to a mirror that makes it work 'by luck' ?

Because I can now try a 'volta fetch node@20' and it does nothing, ctrl+c, arrow up, retry, and nothing" and I have been doing that for the last 15 minutes. status.nodejs.org says everything is fine.

TheJaredWilcurt commented 8 months ago

Last week I convinced a co worker to try out Volta, she had to disconnect from the VPN for it to work, too much of an inconvenience for her to continue using it.

Just now my GHA failed due to Volta not being able to download Node, while the Node servers are fine, like others above. My GHA with Volta worked dozens of times before, and after waiting 3-5 minutes and re-running the failed CI, it worked again. Error from the failed attempt:

Volta error: Could not unpack Node v21.1.0

Please ensure the correct version is specified.
Volta error: Error cause: failed to unpack `/opt/hostedtoolcache/volta/1.1.1/x64/tmp/.tmpxEuBAO/node-v21.1.0-linux-x64/bin/node`

Error cause: failed to unpack `/opt/hostedtoolcache/volta/1.1.1/x64/tmp/.tmpxEuBAO/node-v21.1.0-linux-x64/bin/node`

Error cause: failed to unpack `node-v21.1.0-linux-x64/bin/node` into `/opt/hostedtoolcache/volta/1.1.1/x64/tmp/.tmpxEuBAO/node-v21.1.0-linux-x64/bin/node`

Error cause: Resource temporarily unavailable (os error 11)
Error: Process completed with exit code 126.

I'm really hoping these problems can be resolved soon, especially automatically. If Volta could fall back to a randomly selected mirror from a list automatically, then users won't even know they hit a rate-limit and Volta would continue trying mirrors until it finds one that works, so it just takes a little longer to get the download started, but other than that works normally. Obviously would need to double-check a hash before accepting the file from the mirror, but other than that this could all be done behind the scenes.

jorismak commented 8 months ago

Instead of installing volta in every action as a step, I now do it in a prepared docker image. So volta is pre installed in our images used for actions/pipelines. And the config is changed to use the npm mirrors as much as possible. Since then , no more issues as far as I know.

In a bigger version of our pipeline docker images I even run a 'volta fetch @.***' and then the same for 16, 18 and 20. Those docker images are rebuilt every week on a schedule.

So as long as the devs that use them have the projects pinned to the latest version of a certain node major, the docker image had them available already and often works without having an internet connection at all .

It was our fallback plan because hlthe interaction between volta and the node servers was preventing devs from releasing, apparently

On Wed, Jan 24, 2024, 15:26 The Jared Wilcurt @.***> wrote:

Last week I convinced a co worker to try out Volta, she had to disconnect from the VPN for it to work, too much of an inconvenience for her to continue using it.

Just now my GHA failed due to Volta not being able to download Node, while the Node servers are fine, like others above.

Volta error: Could not unpack Node v21.1.0

Please ensure the correct version is specified. Volta error: Error cause: failed to unpack /opt/hostedtoolcache/volta/1.1.1/x64/tmp/.tmpxEuBAO/node-v21.1.0-linux-x64/bin/node

Error cause: failed to unpack /opt/hostedtoolcache/volta/1.1.1/x64/tmp/.tmpxEuBAO/node-v21.1.0-linux-x64/bin/node

Error cause: failed to unpack node-v21.1.0-linux-x64/bin/node into /opt/hostedtoolcache/volta/1.1.1/x64/tmp/.tmpxEuBAO/node-v21.1.0-linux-x64/bin/node

Error cause: Resource temporarily unavailable (os error 11) Error: Process completed with exit code 126.

Really hoping these problems can be resolved soon, especially automatically. If Volta could fall back to a randomly selected mirror from a list automatically, then users won't even know they hit a rate-limit and Volta would continue trying mirrors until it finds one that works, so it just takes a little longer to get the download started, but other than that works normally. Obviously would need to double-check a hash before accepting the file from the mirror, but other than that this could all be done behind the scenes.

— Reply to this email directly, view it on GitHub https://github.com/volta-cli/volta/issues/1523#issuecomment-1908236258, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMD2DUHT5D6E3CX3FIKKRDYQEKXZAVCNFSM6AAAAAAZVJOMKCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBYGIZTMMRVHA . You are receiving this because you commented.Message ID: @.***>

jorismak commented 6 months ago

On Sun, 3 Mar 2024 at 09:40, codercoin @.***> wrote:

same problem.😕

We've used docker images in pipelines now for more than a few months, which have Volta included from the start. So the 'volta installer' is run during creation of the docker image, and then the environment variables are set in the image so that it's ready to use when a container is started.

In the home folder of the container, we create a file ~/.volta/hooks.json with the following contents:

{
    "node": {
        "index": {
            "prefix": "https://npmmirror.com/mirrors/node/"
        },
        "distro": {
            "template": "
https://npmmirror.com/mirrors/node/v{{version}}/{{filename}}"
        }
    },
    "npm": {
        "index": {
            "prefix": "https://registry.npmmirror.com/"
        },
        "distro": {
            "template": "https://registry.npmmirror.com/npm/-/{{filename}}"
        }
    },
    "pnpm": {
        "index": {
            "prefix": "https://registry.npmmirror.com/"
        },
        "distro": {
            "template": "https://registry.npmmirror.com/pnpm/-/{{filename}}"
        }
    }
}

This tells Volta to use the npmmirror for downloading node / npm. Although the response time seems a bit slower (latency), download speeds are fast and we had no problems since then.

NodeJS is working on a new site, though.

Barabba11 commented 2 months ago

"volta" "install" "node" Volta v1.1.1

Could not download Node version registry from https://mirrors.dotsrc.org/nodejs/release/index.json

Please verify your internet connection.

Error cause: Io Error: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

Oi oi oi.. it attempts to wownload from https and it is not able to do it?

DobyAsa commented 1 week ago

I used this for the ~/.volta/hooks.json and it seems to work too:我使用这个用于~/.volta/hooks.json，它似乎也有效：

{
  "node": {
    "index": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/index.json"
    },
    "distro": {
      "template": "https://mirrors.dotsrc.org/nodejs/release/v{{version}}/node-v{{version}}-{{os}}-{{arch}}.tar.gz"
    }
  }
}

Thank you!!! This method works for me!