Need better sig-fail diagnostics by default. "warning: Signature verification failed" is insufficient.

saghm commented 4 years ago

When we verify the signature, we 'correctly' simply report success/failure, however it may be of value to report more detail by default so that if users encounter the issue seen by the OP, we can more easily diagnose the problem as often these things are transient and hard to reproduce.

ORIGINAL ISSUE

Problem I just ran rustup update locally, and I received a warning that "signature verification failed" when downloading what appears to be the manifest for the nightly channel versions. This is the warning I received:

warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-nightly.toml'

And here is the entire output from running rustup update:

info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-nightly.toml'
info: latest update on 2020-08-18, rust version 1.47.0-nightly (792c645ca 2020-08-17)
info: downloading component 'rustfmt'
info: downloading component 'rust-src'
info: downloading component 'rust-std'
info: downloading component 'rustc'
 51.5 MiB /  51.5 MiB (100 %)  32.6 MiB/s in  1s ETA:  0s
info: downloading component 'clippy'
info: downloading component 'cargo'
info: removing previous version of component 'rustfmt'
info: removing previous version of component 'rust-src'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'rustc'
info: removing previous version of component 'clippy'
info: removing previous version of component 'cargo'
info: installing component 'rustfmt'
info: Defaulting to 500.0 MiB unpack ram
info: installing component 'rust-src'
info: installing component 'rust-std'
 20.7 MiB /  20.7 MiB (100 %)  13.4 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 51.5 MiB /  51.5 MiB (100 %)  14.7 MiB/s in  3s ETA:  0s
info: installing component 'clippy'
info: installing component 'cargo'

  stable-x86_64-unknown-linux-gnu unchanged - rustc 1.45.2 (d3fb005a3 2020-07-31)
   nightly-x86_64-unknown-linux-gnu updated - rustc 1.47.0-nightly (792c645ca 2020-08-17) (from rustc 1.47.0-nightly (7e6d6e5f5 2020-08-16))

info: cleaning up downloads & tmp directories

Steps I unfortunately have not able to reproduce this bug. Running rustup update again did not give the same warning, nor did removing the nightly toolchain and installing from scratch again. I also tried moving my ~/.rustup directory to somewhere else and running rustup instlal nightly again, but I didn't get the warning that time either. I'm a little hesitant to completely uninstall rustup and install everything from scratch again, so I figured I would wait until I heard back on this issue to see if that would be useful in some way.

Notes

This occurred on an Arch Linux box with rustup installed through the package manager.

Output of rustup --version:

rustup 1.22.1 (2020-07-08)

Output of rustup show:

Default host: x86_64-unknown-linux-gnu
rustup home:  /home/saghm/.rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu (default)
nightly-x86_64-unknown-linux-gnu
1.43.1-x86_64-unknown-linux-gnu

installed targets for active toolchain
--------------------------------------

x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl

active toolchain
----------------

stable-x86_64-unknown-linux-gnu (default)
rustc 1.45.2 (d3fb005a3 2020-07-31)

kinnison commented 4 years ago

Interesting, I should likely increase the diagnostics on that -- we're currently opportunistically checking the GPG signature as part of the install process, I wonder if you managed to catch it before it was updated or somesuch.

At this point, there's nothing you can do to make it easier for me to tell what happened, so I'm going to repurpose this issue.

kinnison commented 4 years ago

This ought to be straightforward to do, (though not easy to decide what to report). I'd be happy to chat to someone about resolving it.

sidkshatriya commented 1 year ago

@saghm

warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-nightly.toml'

My guess is if you're running rustup in a VM, this signature verification could fail due to clock skew. Make sure your VM time is correct and see if that helps...

saghm commented 1 year ago

I honestly don't remember exactly what the circumstances were when I ran into this issue (I forgot that I even created this issue!), but based on the fact that I reported it as occurring on Arch Linux, I'm fairly certain it was not in a VM, as I boot Arch natively on all my personal machines and haven't ever used it in a VM. I don't remember ever having run into this again since then, so it could have been an OS bug or even a hardware issue.

Nashenas88 commented 1 year ago

I just got this now:

warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-nightly.toml'
info: latest update on 2023-02-01, rust version 1.69.0-nightly (dc1d9d50f 2023-01-31)

joshhansen commented 1 year ago

Same:

$ rustup toolchain add nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-nightly.toml'
info: latest update on 2023-02-01, rust version 1.69.0-nightly (dc1d9d50f 2023-01-31)

Seems weird to me that signature verification failures are just a warning?

UPDATE

Beta channel also fails verification:

$ rustup check
stable-x86_64-unknown-linux-gnu - Up to date : 1.67.0 (fc594f156 2023-01-24)
warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-beta.toml'
beta-x86_64-unknown-linux-gnu - Update available : 1.66.0-beta.1 (e080cc5a6 2022-11-01) -> 1.68.0-beta.1 (efd27454a 2023-01-25)
rustup - Up to date : 1.25.1

I'm not running in a VM

sidkshatriya commented 1 year ago

I'm not running in a VM

OK. BTW is your system time and date correct?

tshepang commented 1 year ago

I got same, and...


❯ timedatectl
               Local time: Wed 2023-02-01 11:06:50 SAST
           Universal time: Wed 2023-02-01 09:06:50 UTC
                 RTC time: Wed 2023-02-01 09:06:50
                Time zone: Africa/Johannesburg (SAST, +0200)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

thomasweitzel commented 1 year ago

Same here, runing Ubuntu 22.04 with latest updates applied (no VM):

$ timedatectl
               Local time: Mi 2023-02-01 10:18:13 CET
           Universal time: Mi 2023-02-01 09:18:13 UTC
                 RTC time: Mi 2023-02-01 09:18:13
                Time zone: Europe/Berlin (CET, +0100)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

sidkshatriya commented 1 year ago

Sorry, for me the error got fixed once I made sure my system date/time was correct. Not sure what is happening for you. Try making sure your system packages in your distribution are upto date also... perhaps that might help?

P.S. I just tried rustup myself and I'm getting the same problem also. My earlier response in this comment was related to an older signature verification problem.

See at the end of the ticket for a more useful response related to the issues being faced today.

tshepang commented 1 year ago

I just updated my Debian testing to make sure, even rebooted, and checked time (all good), and still experiencing same problem.

Rob2309 commented 1 year ago

Consistently getting this error currently on windows with correct time/date.

g2p commented 1 year ago

See #3185 (and https://github.com/rust-lang/simpleinfra/issues/218) for failures that started occuring today.

rjwalsh commented 1 year ago

I'm also seeing this on macOS. My clock seems fine, although that's not based on my actually doing anything to verify that.

~ $ rustup update
info: syncing channel updates for 'stable-aarch64-apple-darwin'
warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-stable.toml'
info: latest update on 2023-01-26, rust version 1.67.0 (fc594f156 2023-01-24)
info: downloading component 'rls'
info: downloading component 'rust-src'
info: downloading component 'rust-analysis'
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-std'
 27.4 MiB /  27.4 MiB (100 %)  19.6 MiB/s in  1s ETA:  0s
info: downloading component 'rustc'
 55.7 MiB /  55.7 MiB (100 %)  18.0 MiB/s in  3s ETA:  0s
info: downloading component 'rustfmt'
info: removing previous version of component 'rls'
info: removing previous version of component 'rust-src'
info: removing previous version of component 'rust-analysis'
info: removing previous version of component 'cargo'
info: removing previous version of component 'clippy'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'rustc'
info: removing previous version of component 'rustfmt'
info: installing component 'rls'
info: installing component 'rust-src'
info: installing component 'rust-analysis'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-std'
 27.4 MiB /  27.4 MiB (100 %)  18.5 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 55.7 MiB /  55.7 MiB (100 %)  20.6 MiB/s in  2s ETA:  0s
info: installing component 'rustfmt'
info: syncing channel updates for 'nightly-aarch64-apple-darwin'
warning: Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-nightly.toml'
info: latest update on 2023-02-04, rust version 1.69.0-nightly (658fad6c5 2023-02-03)
info: downloading component 'rust-src'
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
 19.3 MiB /  19.3 MiB (100 %)  18.1 MiB/s in  1s ETA:  0s
info: downloading component 'rust-std'
 27.4 MiB /  27.4 MiB (100 %)  17.6 MiB/s in  1s ETA:  0s
info: downloading component 'rustc'
 55.9 MiB /  55.9 MiB (100 %)  18.0 MiB/s in  3s ETA:  0s
info: downloading component 'rustfmt'
info: removing previous version of component 'rust-src'
info: removing previous version of component 'cargo'
info: removing previous version of component 'clippy'
info: removing previous version of component 'rust-docs'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'rustc'
info: removing previous version of component 'rustfmt'
info: installing component 'rust-src'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-docs'
 19.3 MiB /  19.3 MiB (100 %)   6.5 MiB/s in  1s ETA:  0s
info: installing component 'rust-std'
 27.4 MiB /  27.4 MiB (100 %)  18.6 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 55.9 MiB /  55.9 MiB (100 %)  20.5 MiB/s in  2s ETA:  0s
info: installing component 'rustfmt'
info: checking for self-updates
info: downloading self-update

   stable-aarch64-apple-darwin updated - rustc 1.67.0 (fc594f156 2023-01-24) (from rustc 1.66.1 (90743e729 2023-01-10))
  nightly-aarch64-apple-darwin updated - rustc 1.69.0-nightly (658fad6c5 2023-02-03) (from rustc 1.68.0-nightly (52372f9c7 2023-01-21))

info: cleaning up downloads & tmp directories

sidkshatriya commented 1 year ago

Try updating your rustup itself and try again.

$ rustup self update
$ rustup update

See https://github.com/rust-lang/rustup/pull/3186 for the specific PR that fixes this current issue.

bjorn3 commented 1 year ago

The rustup invocation @rjwalsh posted has already updated rustup, so it should be fixed for future rustup invocations.

info: checking for self-updates
info: downloading self-update

aricooperman commented 1 year ago

I also get this now consistently on Fedora 37. Ran self update as well and my date/time is accurate

bjorn3 commented 1 year ago

Did you install rustup using the distro package manager? If so self updates are disabled and you need to wait for fedora to push an update.

aricooperman commented 1 year ago

I did not, ran the script from https://rustup.rs/

On Mon, Feb 6, 2023 at 2:13 AM bjorn3 @.***> wrote:

Did you install rustup using the distro package manager? If so self updates are disabled and you need to wait for fedora to push an update.

— Reply to this email directly, view it on GitHub https://github.com/rust-lang/rustup/issues/2462#issuecomment-1418750647, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABER2XA6PUM2VRLIFJWXE5LWWC6E3ANCNFSM4QELVWEA . You are receiving this because you commented.Message ID: @.***>

bjorn3 commented 1 year ago

What is the full output of rustup self update? For me it was

info: checking for self-updates
info: downloading self-update
  rustup updated - 1.25.2 (from 1.25.1)

the first time and

info: checking for self-updates
  rustup unchanged - 1.25.2

the second time.

aricooperman commented 1 year ago

I don't have the first run but running it now I get this:

$ rustup self update info: checking for self-updates rustup unchanged - 1.25.2

On Mon, Feb 6, 2023 at 9:45 AM bjorn3 @.***> wrote:

What is the full output of rustup self update? For me it was

info: checking for self-updates info: downloading self-update rustup updated - 1.25.2 (from 1.25.1)

the first time and

info: checking for self-updates rustup unchanged - 1.25.2

the second time.

— Reply to this email directly, view it on GitHub https://github.com/rust-lang/rustup/issues/2462#issuecomment-1419392168, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABER2XBV3FUH6Q6YS36PM6DWWETA7ANCNFSM4QELVWEA . You are receiving this because you commented.Message ID: @.***>

bjorn3 commented 1 year ago

1.25.2 should be the fixed version. Weird.

aricooperman commented 1 year ago

Sorry, i think I was unclear. I ran the self update after first doing the update that showed the warnings.

I just re-ran update with this version and I see no warnings now

On Mon, Feb 6, 2023 at 1:02 PM bjorn3 @.***> wrote:

1.25.2 should be the fixed version. Weird.

— Reply to this email directly, view it on GitHub https://github.com/rust-lang/rustup/issues/2462#issuecomment-1419669063, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABER2XHIX5EGOSUIIFBTBXDWWFKDTANCNFSM4QELVWEA . You are receiving this because you commented.Message ID: @.***>

ohir commented 1 year ago

And again: "Signature verification failed for 'https://static.rust-lang.org/dist/channel-rust-stable.toml". It was from 1.25.1, but I am reporting it here, because it still was a warning.

Establishing the signature to be invalid and then proceeding anyway can not be described in a politically correct way, so I'll spare.

Just note that docs saying straight that "We will happily install on your machine whatever we or someone else meantime put at the 'https://static.rust-lang.org/dist/" would be better security-wise than checking the signature then continue regardless of this check result.

kpcyrd commented 1 year ago

This issue is still waiting on somebody from the Rust org to regenerate the self-signatures with sha256 or sha512. #3185, https://github.com/rust-lang/simpleinfra/issues/218

The following two signatures need to be regenerated:

The subkey binding signature
Self-certification of a UID

rust-lang / rustup

Need better sig-fail diagnostics by default. "warning: Signature verification failed" is insufficient. #2462