search

rust-lang / cargo

The Rust package manager
https://doc.rust-lang.org/cargo
Apache License 2.0
12.78k stars 2.42k forks source link

Cargo build failed with spurious network error #6513

Open weiby3 opened 5 years ago

weiby3 commented 5 years ago

Problem error: failed to load source for a dependency on rand

Steps

$ cargo run --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
warning: spurious network error (2 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
error: failed to load source for a dependency on `rand`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: 无法与服务器建立连接
; class=Os (2)

The chinese sentences above are "Unable to establish connection to server". I have check my firewall, cargo isn't blocked, and there is personal CA installed by Adguard, no proxy.

Notes OS: Windows 7 64bit, mingw64 with MSYS2 Output of cargo version: cargo 1.29.0

alexcrichton commented 5 years ago

Hm is this perhaps a network-level issue where github.com or github.com/rust-lang is blocked? Otherwise I'm not sure how we'd fix this as it seems Cargo can't connect :(

weiby3 commented 5 years ago

I try git clone and curl https://github.com/rust-lang and they works fine. On my computer github.com can be accessed successfully, I have search the firewall and there aren't any block rule related to it. Maybe these are some useful information. I use HUORONG SECURITY as my anti-virus software, and if there is a program trying to connect to Internet, it will popup a windows and ask for permission. Strangely when I run cargo run --verbose there is no popup window, but if I run curl https://github.com/rust-lang there is a popup window, so it seems Cargo can't connect. Is it possible to get more error information?

alexcrichton commented 5 years ago

Hm so AFAIK there's a few different ways to touch the network on Windows, and it may be that Cargo, through libgit2, is using a different way that's may blocked or doesn't provide a popup? Can you temporarily disable your firewall/virus scanning to see if that fixes the issue?

ddw360 commented 5 years ago

@alexcrichton Disabling antivirus, firewalls, etc - don't seem to work. I was able to do a cargo build in Windows 10 x64 but not in Windows Subsystem for Linux. Both have versions re: nightly, etc. The build fails when trying to retrieve from github dependencies. Command issued was "cargo build --release --target=wasm32-unknown-unknown"

alexcrichton commented 5 years ago

Hm interesting! So this probably isn't a firewall/antivirus issue if you've disabled all those and it still didn't work. It works on Windows though and not the WSL? What we use on Linux is far more bog-standard I think than the Windows implementation, so I'm surprised it wouldn't work there if it works on Windows... I wonder if that's a WSL bug?

ddw360 commented 5 years ago

WSL environment info: Linux ZEPHYR 4.4.0-17134-Microsoft #471-Microsoft Fri Dec 07 20:04:00 PST 2018 x86_64 x86_64 x86_64 GNU/Linux PRETTY_NAME="Ubuntu 16.04.5 LTS" Architecture: x86_64

Don't know if it matters or not - but here are contents of Cargo.toml below and the git repos that the cargo build command is trying to pull for dependencies. when "cargo build --release --target=wasm32-unknown-unknown" is issued. Fails on trying to resolve the first "hdk" dependency.

[package] name = "chat" version = "0.1.0" authors = ["willem willemolding@gmail.com"] edition = "2018" [dependencies] serde = "1.0" serde_json = "1.0" serde_derive = "1.0" machine-ip = "0.2.1" hdk = { git = "https://github.com/holochain/holochain-rust" , branch = "master" } holochain_wasm_utils = { git = "https://github.com/holochain/holochain-rust" , branch = "master" } holochain_core_types_derive = { git = "https://github.com/holochain/holochain-rust" , branch = "master" } [lib] path = "src/lib.rs" crate-type = ["cdylib"]

alexcrichton commented 5 years ago

Can you perhaps try strace? That may help illuminate which syscall is failing and may help tracking this down?

ddw360 commented 5 years ago

@alexcrichton https://pastebin.com/uqcrHrzT

weiby3 commented 5 years ago

Sorry for late response. I have tried to disable firewall and anti-virus software, but it still has this problem. Is it possible to expose the download directory filename and/or commands? Maybe try to do it by myself can solve this problem.

ddw360 commented 5 years ago

@weiby3 @alexcrichton Weiby3 - I am curious if you are in actual Windows standard command line, powershell or a WSL environment when encountering issues. I haven't tested doing things in Powershell but do get the spurious network error in WSL.

weiby3 commented 5 years ago

I have tried in cmd.exe and got the same problem. Windows 7 doesn't have WSL, so I can't try. Powershell also not work.

alexcrichton commented 5 years ago

@ddw360 do you perhaps have a proxy configured? It looks like an ntlm proxy or something like that, running on port 3128

ddw360 commented 5 years ago

@alexcrichton don't think so - here is my evidence:

  1. from Windows cmd prompt: netstat -abno | findstr /R "^.:3128.$" --> no results

  2. Internet Explorer Settings - no proxy being used for internet connection

  3. whatismyipaddress: Hostname: | cpe-76-170-98-202.socal.res.rr.com Services: None detected Proxy server not detected. Broadband Dynamic IP

  4. netstat from bash (ubuntu 16.04) shows no use of proxy port 3128

  5. /etc/resolv.conf shows no use of proxy - nameservers are all gateway addresses and then gateway pointed to publ ic dns/internet

  6. git version is 2.7.4 (<< i am going to try to get a later version of git by compiling from source and see if that does the trick since cargo relies on git - since current ubuntu is 16.04).

I am going to try to move to WSL ubuntu 18.04 LTS and a higher git version and see if that changes things at all.

weiby3 commented 5 years ago

If I try to run cargo via a proxy, it can "Updating registry https://github.com/rust-lang/crates.io-index", but it will get the following output:

$ https_proxy=socks5://127.0.0.1:1080 cargo run --verbose
 Downloading winapi-x86_64-pc-windows-gnu v0.4.0
error: unable to get packages from source

Caused by:
  [35] SSL connect error (schannel: failed to receive handshake, SSL/TLS connection failed)

Othrtwise, I try to install rust with msvc, it can download and build successfully.

c:\msys64\tmp\rust\guessing_game>cargo run
   Compiling winapi v0.3.6
   Compiling libc v0.2.46
   Compiling rand v0.4.3
   Compiling rand v0.3.22
   Compiling guessing_game v0.1.0 (C:\msys64\tmp\rust\guessing_game)
    Finished dev [unoptimized + debuginfo] target(s) in 7.72s
     Running `target\debug\guessing_game.exe`
Guess the number!
Please input your guess.
123
You guessed: 123

UPDATE: After download rust package via proxy, I run cargo without proxy, and it build successfully. Here are all the commands I have tried and their output:

$ cargo run --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
warning: spurious network error (2 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
error: failed to load source for a dependency on `rand`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: 无法与服务器建立连接
; class=Os (2)
$ https_proxy=socks5://127.0.0.1:1080 cargo run --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading rand v0.3.22
error: unable to get packages from source

Caused by:
  [35] SSL connect error (schannel: failed to receive handshake, SSL/TLS connection failed)
https_proxy=socks5://127.0.0.1:1080 cargo run --verbose
 Downloading rand v0.3.22
error: unable to get packages from source

Caused by:
  [35] SSL connect error (schannel: failed to receive handshake, SSL/TLS connection failed)
$ cargo run --verbose
   Compiling winapi-x86_64-pc-windows-gnu v0.4.0
   Compiling winapi v0.3.6
   Compiling libc v0.2.46
     Running `rustc --crate-name build_script_build 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\build.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=cd7a6ff3618fa678 -C extra-filename=-cd7a6ff3618fa678 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-x86_64-pc-windows-gnu-cd7a6ff3618fa678' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `rustc --crate-name build_script_build 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.6\build.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 --cfg 'feature="minwindef"' --cfg 'feature="ntsecapi"' --cfg 'feature="profileapi"' --cfg 'feature="winnt"' -C metadata=db0e30f3a93487bc -C extra-filename=-db0e30f3a93487bc --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-db0e30f3a93487bc' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `rustc --crate-name build_script_build 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.46\build.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="use_std"' -C metadata=f439455b17189f8e -C extra-filename=-f439455b17189f8e --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\build\libc-f439455b17189f8e' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-x86_64-pc-windows-gnu-cd7a6ff3618fa678\build-script-build`
     Running `C:\msys64\tmp\rust\guessing_game\target\debug\build\libc-f439455b17189f8e\build-script-build`
     Running `rustc --crate-name winapi_x86_64_pc_windows_gnu 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=b1bde479814dcaee -C extra-filename=-b1bde479814dcaee --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
     Running `C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-db0e30f3a93487bc\build-script-build`
     Running `rustc --crate-name libc 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.46\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="use_std"' -C metadata=df7ade2be09a8644 -C extra-filename=-df7ade2be09a8644 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `rustc --crate-name winapi 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.6\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="minwindef"' --cfg 'feature="ntsecapi"' --cfg 'feature="profileapi"' --cfg 'feature="winnt"' -C metadata=7a3aa7d7c95390c7 -C extra-filename=-7a3aa7d7c95390c7 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'winapi_x86_64_pc_windows_gnu=C:\msys64\tmp\rust\guessing_game\target\debug\deps\libwinapi_x86_64_pc_windows_gnu-b1bde479814dcaee.rlib' --cap-lints allow --cfg 'feature="sspi"' --cfg 'feature="wincred"' --cfg 'feature="lsalookup"' --cfg 'feature="excpt"' --cfg 'feature="ktmtypes"' --cfg 'feature="basetsd"' --cfg 'feature="guiddef"' --cfg 'feature="vcruntime"' --cfg 'feature="subauth"' --cfg 'feature="windef"' --cfg 'feature="ntdef"' -l dylib=winapi_advapi32 -l dylib=winapi_credui -l dylib=winapi_kernel32 -l dylib=winapi_secur32 -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
   Compiling rand v0.4.3
     Running `rustc --crate-name rand 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\rand-0.4.3\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="libc"' --cfg 'feature="std"' -C metadata=525ab3e86af69877 -C extra-filename=-525ab3e86af69877 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'winapi=C:\msys64\tmp\rust\guessing_game\target\debug\deps\libwinapi-7a3aa7d7c95390c7.rlib' --cap-lints allow -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
   Compiling rand v0.3.22
     Running `rustc --crate-name rand 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\rand-0.3.22\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=028fc22332238648 -C extra-filename=-028fc22332238648 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'libc=C:\msys64\tmp\rust\guessing_game\target\debug\deps\liblibc-df7ade2be09a8644.rlib' --extern 'rand=C:\msys64\tmp\rust\guessing_game\target\debug\deps\librand-525ab3e86af69877.rlib' --cap-lints allow -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
   Compiling guessing_game v0.1.0 (file:///C:/msys64/tmp/rust/guessing_game)
     Running `rustc --crate-name guessing_game 'src\main.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=0ad3830cad379b28 -C extra-filename=-0ad3830cad379b28 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -C 'incremental=C:\msys64\tmp\rust\guessing_game\target\debug\incremental' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'rand=C:\msys64\tmp\rust\guessing_game\target\debug\deps\librand-028fc22332238648.rlib' -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
    Finished dev [unoptimized + debuginfo] target(s) in 16.01s
     Running `target\debug\guessing_game.exe`
Guess the number!
Please input your guess.
123
You guessed: 123
ddw360 commented 5 years ago

@alexcrichton upgrading to WSL ubuntu 18.04, git 2.20.1, lastet rust and cargo nightly has not resolved. Could this be related to: https://github.com/rust-lang/cargo/issues/1180# in any way?

alexcrichton commented 5 years ago

This doesn't look related to certificate validation just yet, but your examples are also somewhat hard to follow for me :(. What caused the packages to actually be downloaded? It looks like in some cases connections to crates.io are disallowed but github is allowed. (I'm not really sure what's going on here...)

weiby3 commented 5 years ago

The project comes from a tutorial. The following is Cargo.toml.

[package]
name = "guessing_game"
version = "0.1.0"
authors = ["account"]

[dependencies]
rand = "0.3.14"
ddw360 commented 5 years ago

@alexcrichton I am trying to run the holochain stack and an example project See: https://developer.holochain.org/start.html https://github.com/holochain/holochat-rust

The "hc package" and "hc test" commands issue the following "cargo": "cargo build --release --target=wasm32=unknown-unknown"

and it fails - with SSL related issues - per the pastebin's I posted.

For me - what I have is problems specifically in Windows Subsystem for Linux. And pathing to cargo or hc are not the problem - they both come up correctly in bash.

In Windows 10 cmd.exe and powershell - no problems once I made sure that cargo and rustc were properly in the system path.

@weiby3 is likely having different root cause? But same resulting error - re: SSL and Spurious and git?

d3sbar commented 5 years ago

@alexcrichton I am trying different projects in Windows Ubuntu 18.04 WSL - i.e. actix/examples. So cloned down the repo, go to the "hello-world" example and do cargo run. Get spurious network error. Firewall is turned off, no antivirus running, etc.

$ cargo build
    Updating crates.io index
warning: spurious network error (2 tries remaining): [7] Couldn't connect to server; class=Net (12)
warning: spurious network error (1 tries remaining): [7] Couldn't connect to server; class=Net (12)
error: failed to load source for a dependency on `actix`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [7] Couldn't connect to server; class=Net (12)
differui commented 5 years ago

@d3sbar I stuck in same error. Fix that by removing [http_proxy] field in ~/.cargo/config.

WindSoilder commented 5 years ago

Got the same issue while Updating crate.io index... The error message is: 

warning: spurious network error (2 tries remaining): HTTP parser error: stream ended at an unexpected time; class=Net (12)

But I think it's just caused by my Network problem...

I use wireshark to capture segments between my host and github(The ip address is 13.229.188.59).

From wireshark, I find the following phenomena:

  1. github have tried to send many Retransmission for the same segment and,
  2. my host is sending and sending back duplicate ACK.

It seems that my ACK can't reach github machine in time It seems that it's just cause by network problem(like network congestion)

IMO, if there are stable Crates.io MIRROR exists in my region(Asia), the problem can be solved(by editing ~/.cargo/config).

Hope this information can help..

My OS: Mac 10.14.6 Cargo version: 1.36.0

baburkin commented 5 years ago

I got the same issue on Windows 7 and it looks like the reason behind that is cargo cannot find its way through our corporate NTLM proxy.

I was troubleshooting this with plain curl. When I try to run 

curl https://github.com/rust-lang/crates.io-index

I get the response 

curl: (56) Received HTTP code 407 from proxy after CONNECT

which implies that I need to authenticate in the proxy. By using curl with -vv option I can see that it tries to authenticate using NTLM:

...
Proxy-Authenticate: Negotiate
Proxy-Authenticate: NTLM

* Ignore NNNN bytes of response-body
* Received HTTP code 407 from proxy after CONNECT
* Closing connection 0
curl: (56) Received HTTP code 407 from proxy after CONNECT

Then what I did I added the following environment variables via Control Panel (the addresses are distorted on purpose):

HTTP_PROXY=http://ntlm-proxy.org.com:8080
HTTPS_PROXY=http://ntlm-proxy.org.com:8080
PROXY_USER=NT_DOMAIN\username
PROXY_PASS=the_password  <-- I know, not ideal, but see below

After that I was able to receive the crates index using the following command:

curl -vv --proxy-any -U : https://github.com/rust-lang/crates.io-index

It worked, it also worked with --proxy-ntlm instead of --proxy-any, but if I don't specify this parameter, I get the 407 error again. Instead of adding environment vars for user and password, I also was able to specify --proxy-user NT_DOMAIN\username and provide the password interactively - and it also worked.

So it looks like at the moment rust's cargo lacks support for NTLM proxy specifics, including authentication. Given that curl does support this, I think it is just a matter of adding this support in future versions.

Ideally, it would be nice to be able to to specify in ~/.cargo/config:

[http]
proxy = "host:port" # HTTP proxy to use for HTTP requests (defaults to none)
proxy-auth = "ntlm" # negotiate, basic, digest, etc... 
proxy-user = "username"
proxy-password = "password"
...
alexcrichton commented 5 years ago

Thanks for the investigation @baburkin!

This is unfortunately a pretty long a sprawling issue, mind opening a focused issue purely on supporting proxy authentication (aka NTLM)?

baburkin commented 5 years ago

@alexcrichton - created Issue 7330 as a feature request. Would love to contribute.

iav commented 5 years ago

Have similar error trying to do a cross-compilation in docker. I got it in automatic builder on a Docker Hub, then on a wps in Amazon S3. This dockerfile (scenario) works on my pc. And it's not a "simple" network error, like dns not work, site unreacheable, proxy should be used, proxy auth etc. Problem occurs on machine free internet access, and target sites reacheable from same sessions with wget

error: (in docker container)

cargo --version
cargo 1.37.0 (9edd08916 2019-08-02)

uname -a
Linux b584dc1b7555 5.0.0-1012-aws #13-Ubuntu SMP Fri Aug 2 12:25:32 UTC 2019 armv7l GNU/Linux

# qemu-arm-static --version
qemu-arm version 4.1.0 (qemu-4.1.0-1.fc31)

outside container —  aws instance:
$ uname -a
Linux ip-172-30-0-82 5.0.0-1012-aws #13-Ubuntu SMP Fri Aug 2 12:25:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

# cargo run --verbose
    Updating crates.io index
warning: spurious network error (2 tries remaining): could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)
warning: spurious network error (1 tries remaining): could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)

Start of Dockerfile to reproduce inside AWS t2.medium instance, image id ubuntu/images/hvm-ssd/ubuntu-disco-19.04-amd64-server-20190819 (ami-00290d66f0da25f73)

FROM multiarch/qemu-user-static as qemu
FROM arm32v7/rust:1.37-buster as rust
COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
WORKDIR /
RUN git clone https://github.com/rustapp.git app
WORKDIR /app
RUN cargo build
iav commented 5 years ago

before run cross-platform image, qemu need to be added to system. Fast and easy for container runner docker run --rm --privileged multiarch/qemu-user-static:register --reset

ehuss commented 5 years ago

@iav can you file a new issue, that seems unrelated to this one.

iav commented 5 years ago

ok.

kasvith commented 5 years ago

The issue remains the same in my Windows 10 x64 setup too.

I have not set up proxies either.

toolchain stable-x86_64-pc-windows-msvc cargo version is cargo 1.38.0 (23ef9a4ef 2019-08-20)

$ cargo run
    Updating crates.io index
warning: spurious network error (2 tries remaining): failed to send request: The server name or address could not be resolved
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: The server name or address could not be resolved
; class=Os (2)
error: failed to load source for a dependency on `tokio`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: The server name or address could not be resolved
; class=Os (2)
kamyuentse commented 5 years ago

Same here.

I use Wireshark to monitor the connection between local and cntlm. I notice local(cargo?) reply an Encrypted Alert packet followed by a RST from cntlm.

jasal82 commented 5 years ago

Similar problem here. I'm behind corporate proxy. Cloning the crates.io index from Github works fine but cargo fails with

warning: spurious network error (2 tries remaining): [56] Failure when receiving data from the peer; class=Net (12)
warning: spurious network error (1 tries remaining): [56] Failure when receiving data from the peer; class=Net (12)
error: failed to load source for a dependency on `libc`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [56] Failure when receiving data from the peer; class=Net (12)

There's also a RST in the wireshark log somewhere during the transfer.

baburkin commented 5 years ago

During my investigations on this issue I have found a workaround, which at least works in my network topology with NTLM corporate proxy.

First of all, there are two channels, through which cargo is communicating with crates index. One is libcurl (when accessing crates index file) and the other is libgit2 (when actually doing the subsequent search and fetch from the index). While for the first part your http_proxy / HTTP_PROXY settings at environment level might work, the reason it breaks is actually lying underneath - it breaks when libgit2 code is trying to actually fetch the index repo from GitHub via git protocol.

Some enterprises strictly prohibit access to https://github.com, but reasonable ones enable and control the access via corporate directories, e.g. Active Directory groups, and the user can be authorised when requests are going through NTLM proxy with authentication - authenticated users can be checked against the ACL in Active Directory and allowed access. So the question is how to invoke this authorisation each time a request is made.

If the above is true, one can usually fetch GitHub repos from behind the proxy with a command-line git via configuring proxy in git config (see below).

So while https://github.com/rust-lang/cargo/issues/7330 is still in the works, here's my workaround.

Then that should be it. It worked for me - cargo was finally able to fetch the dependencies. Let me know if it helped in any way.

laurencechan commented 4 years ago

Problem error: failed to load source for a dependency on rand

Steps

$ cargo run --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
warning: spurious network error (2 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
error: failed to load source for a dependency on `rand`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: 无法与服务器建立连接
; class=Os (2)

The chinese sentences above are "Unable to establish connection to server". I have check my firewall, cargo isn't blocked, and there is personal CA installed by Adguard, no proxy.

Notes OS: Windows 7 64bit, mingw64 with MSYS2 Output of cargo version: cargo 1.29.0

Hello, have you fixed the problem?

weiby3 commented 4 years ago

@laurencechan No, it still doesn't work with official source. Currently I use ustc's mirror and it seems no problem yet.

laurencechan commented 4 years ago

@alexcrichton I am trying different projects in Windows Ubuntu 18.04 WSL - i.e. actix/examples. So cloned down the repo, go to the "hello-world" example and do cargo run. Get spurious network error. Firewall is turned off, no antivirus running, etc.

$ cargo build
    Updating crates.io index
warning: spurious network error (2 tries remaining): [7] Couldn't connect to server; class=Net (12)
warning: spurious network error (1 tries remaining): [7] Couldn't connect to server; class=Net (12)
error: failed to load source for a dependency on `actix`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [7] Couldn't connect to server; class=Net (12)

hi, have you fixed this problem?

jasal82 commented 4 years ago

I found the source of the problem in our environment. Obviously the ZScaler proxy modifies the header of some HTTPS responses, specifically the Transfer-Encoding: chunked entry. It replaces it with _ransfer-Encoding: chunked. GnuTLS chokes on this, making the transfer fail. The solution was to rebuild everything with OpenSSL as backend. You can read more about that on https://github.com/curl/curl/pull/3148

GopherJ commented 4 years ago

any workaround?

GopherJ commented 4 years ago 
    Updating crates.io index
warning: spurious network error (2 tries remaining): curl error: Could not resolve host: github.com
; class=Net (12)
warning: spurious network error (1 tries remaining): curl error: Could not resolve host: github.com
; class=Net (12)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  curl error: Could not resolve host: github.com
; class=Net (12)
tymnim commented 4 years ago

I am using zsh on my computer and I had the following error with my "hello world program" because it could not fetch rand = "3.0.0" dependency. Then I opened a new tab, went to my directory and ran cargo build again. It worked fine. I guess in my case it was the new Apple's security policy on Catalina.

JPMoresmau commented 4 years ago

I'm getting this error too today, cannot build anything. It downloads like 30% of the index then fails, takes a few minutes because my connection is not super fast, really frustrating. Can I manually download the index via a browser (something that can recover a dropped connection, for example) and extract it manually? How do I do that?

error: failed to load source for a dependency on `rayon`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to read data: Le serveur a renvoyé une réponse non valide ou non reconnue
; class=Os (2)
taku-n commented 4 years ago

I have had the same problem today but I could fix it.

The way is: Open a file, ~/.gitconfig Found [url "github:"] section? Comment out or delete all of the section like this:

#[url "github:"]
#       InsteadOf = https://github.com/
#       InsteadOf = git@github.com

Have fun!

Reference: https://github.com/rust-lang/cargo/issues/2605 Reference (Japanese): https://kokeshing.com/unable-to-update-registry-error-cargo/

leberknecht commented 4 years ago

Hm, im getting a lot of timeouts atm, but some downloads work, so it doesnt seem to be a network problem per se (all other stuff works fine, browser, npm, composer, pip):

% cargo build
warning: spurious network error (2 tries remaining): [28] Timeout was reached (download of `scopeguard v1.1.0` failed to transfer more than 10 bytes in 30s)
warning: spurious network error (2 tries remaining): [28] Timeout was reached (failed to download any data for `tokio v0.2.13` within 30s)
warning: spurious network error (2 tries remaining): [28] Timeout was reached (download of `generic-array v0.12.3` failed to transfer more than 10 bytes in 30s)
  Downloaded ppv-lite86 v0.2.6
  Downloaded matches v0.1.8
  Downloaded byte-tools v0.3.1
  Downloaded subtle v1.0.0
warning: spurious network error (2 tries remaining): [28] Timeout was reached (failed to download any data for `libc v0.2.68` within 30s)
warning: spurious network error (1 tries remaining): [28] Timeout was reached (download of `libc v0.2.68` failed to transfer more than 10 bytes in 30s)
warning: spurious network error (1 tries remaining): [28] Timeout was reached (failed to download any data for `generic-array v0.12.3` within 30s)
warning: spurious network error (1 tries remaining): [28] Timeout was reached (download of `tokio v0.2.13` failed to transfer more than 10 bytes in 30s)
warning: spurious network error (1 tries remaining): [28] Timeout was reached (failed to download any data for `scopeguard v1.1.0` within 30s)
  Downloaded generic-array v0.12.3
  Downloaded libc v0.2.68
  Downloaded typenum v1.11.2
  Downloaded scopeguard v1.1.0
  Downloaded tokio v0.2.13
  Downloaded mio-uds v0.6.7
warning: spurious network error (2 tries remaining): [28] Timeout was reached (download of `lazy_static v1.4.0` failed to transfer more than 10 bytes in 30s)
warning: spurious network error (2 tries remaining): [28] Timeout was reached (failed to download any data for `iovec v0.1.4` within 30s)
warning: spurious network error (1 tries remaining): [28] Timeout was reached (download of `iovec v0.1.4` failed to transfer more than 10 bytes in 30s)
warning: spurious network error (1 tries remaining): [28] Timeout was reached (failed to download any data for `lazy_static v1.4.0` within 30s)
error: failed to download from `https://crates.io/api/v1/crates/iovec/0.1.4/download`

Caused by:
  [28] Timeout was reached (download of `iovec v0.1.4` failed to transfer more than 10 bytes in 30s)

but https://status.crates.io/ doesnt list any outages.

edit: Maybe that helps someone: i solved the problem by uninstalling rust (was 1.41, shipped through pacman) and installed 1.42 through rustup ( curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh) now the problem is gone

preemeijer commented 4 years ago

I'm experiencing the same problem, today and yesterday, when trying to build a program using cargo build. I'm using cargo 1.40.0 that is coming out of the repo of my Linux distribution.

It always does get a few crates and after a few downloaded crates it will give the error. warning: spurious network error (2 tries remaining): [28] Timeout was reached (download ofrust_decimal v1.3.0failed to transfer more than 10 bytes in 30s)

It does not look like a local network issue. I do think it is a remote cap when trying to get more that nn crates.

Edit What leberknecht has suggested in is edit also worked for me. The warning message is gone. I do get errors regarding openssl but I do not think that this is related to this issue.

ehuss commented 4 years ago

For the recent reports, particularly for people using cargo provided via your Linux package manager, you may want to check which version of libcurl your package manager is using. 7.69.0 had a critical flaw in its HTTP2 handling that causes timeouts. I think 7.69.1 should fix it (or 7.68). See #7974 for more detail.

If you use the pre-built binaries distributed by rustup, they shouldn't have this problem, since they statically link a version of libcurl that does not have that problem.

yihuajack commented 4 years ago

I found that it was indeed a network problem and solved it. If you do not have alternative choice to fetch what you want, you can try using SS or SSR, just go Option Settings (选项设置)->Local Proxy (本地代理), tick "Allow connections from LAN" (允许来自局域网的连接) and set Local Port (本地端口) = 1080. Then append the following to your .zshrc or .bashrc file:

alias proxy="git config --global http.proxy http://127.0.0.1:1080"
alias unproxy="git config --global --unset http.proxy"

Then cargo starts fetching successfully.

Rizary commented 4 years ago

It happened to me also, I'm using simple docker build -t myapp . command. here is my Dockerfile.

FROM rust:1.40-alpine

WORKDIR /usr/src/myapp

COPY . .

RUN cargo install --path .

CMD ["myapp"]
yaleh commented 4 years ago

I have the same issue with WSL. It seems to be related with filesystem.

With a folder mapped to my Windows filesystem:

yale@MATEBOOK-YALE:~/Work/myweb$ /home/yale/.cache/.wrangler/cargo-generate-fc3c677afe428cb1/cargo-generate generate --verbose --git https://github.com/cloudflare/worker-sites-init --name workers-site

 Creating project called `workers-site`...
warning: spurious network error (2 tries remaining): failed to mmap. Could not write data: Permission denied; class=Os (2)
warning: spurious network error (1 tries remaining): failed to mmap. Could not write data: Permission denied; class=Os (2)
Error:  Git Error: failed to clone into: /c/Users/Yale/Work/myweb/workers-sitelxBp3L

With /tmp:

yale@MATEBOOK-YALE:/tmp/myweb$ /home/yale/.cache/.wrangler/cargo-generate-fc3c677afe428cb1/cargo-generate generate --verbose --git https://github.com/cloudflare/worker-sites-init --name workers-site
 Creating project called `workers-site`...
Ignoring: /tmp/myweb/workers-site/.genignore
Ignoring: /tmp/myweb/workers-site/LICENSE_APACHE
Ignoring: /tmp/myweb/workers-site/LICENSE_MIT
 Done! New project created /tmp/myweb/workers-site

There are quite some bugs about mmap in WSL:

phi-gamma commented 4 years ago

I get the mmap version of the issue on Linux:

# CARGO_HOME=/root/some-symlink-to-path/tmp cargo build
    Updating crates.io index
warning: spurious network error (2 tries remaining): failed to mmap. Could not write data: Invalid argument; class=Os (2)
warning: spurious network error (1 tries remaining): failed to mmap. Could not write data: Invalid argument; class=Os (2)
error: failed to get `anyhow` as a dependency of package `some-project v0.1.2 (/mnt/host-src/path/to/project)`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to mmap. Could not write data: Invalid argument; class=Os (2)

# cargo version
cargo 1.43.0
# uname -a
Linux devbox.gwt.local 4.19.104-1.i2n.x86_64 #1 SMP Mon Feb 17 10:19:10 CET 2020 x86_64 x86_64 x86_64 GNU/Linux

It seems to depend on the filesystem as cargo seems to have no issue fetching repos if CARGO_HOME points at tmpfs. The filesystem in question here is 9p with virtio transport.

This is what happens:

23208 execve("/opt/rust/bin/cargo", ["cargo", "build"], 0x7fffff7c6e20 /* 51 vars */) = 0
…
23208 open("/mnt/host-src/path/tmp/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_2Z9JXC", O_RDWR|O_CREAT|O_EXCL, 0600) = 5
… connection setup and about 25 *successful* calls to recvfrom(2)
23208 lseek(5, 4095, SEEK_SET)          = 4095
23208 write(5, "P", 1)                  = 1
23208 mmap(NULL, 11, PROT_READ|PROT_WRITE, MAP_SHARED, 5, 0) = -1 EINVAL (Invalid argument)
…
23208 unlink("/mnt/host-src/path/tmp/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/pack_git2_2Z9JXC") = 0

I. e. the recvfrom(2) calls succeed but mmap(2) on the local fd fails; thus the error message is misleading as this isn’t a networking issue.

phi-gamma commented 4 years ago

Simple reproducer:

# cat m.cc
#include<fcntl.h>
#include<stdio.h>
#include<stdlib.h>
#include<sys/mman.h>

int main (int argc, char **argv) {
    if (argc != 2) { return EXIT_FAILURE; }
    fprintf (stderr, "path=%s\n", argv [1]);
    int fd = open (argv [1], O_RDWR|O_CREAT|O_EXCL, 0600);
    fprintf (stderr, "fd=%d\n", fd);
    if (fd == - 1) { return EXIT_FAILURE; }
    void *mem = mmap (NULL, 11, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
    fprintf (stderr, "mmap=%p\n", mem);
    if (mem == MAP_FAILED) { return EXIT_FAILURE; }
}

# make m && ./m /tmp/foo
g++     m.cc   -o m
path=/tmp/foo
fd=3
mmap=0xf7ed7000

# ./m /mnt/some-9p-mount/bar
path=/mnt/some-9p-mount/bar
fd=3
mmap=0xffffffff

I can only reproduce this on 9p; tmpfs and nfs are fine.

At least, cargo should error out with a clear message stating that CARGO_HOME must reside on an mmap(2)able file system. Also it should be documented in the book as a hard limitation.

The best solution of course would be falling back on something that doesn’t involve mmap(2)ing that file in case the syscall fails.

phi-gamma commented 4 years ago

Fwiw, mounting 9p with mmap=cache fixes the issue. However I still think the error message should indicate the cause of the problem.