rust-lang / rustup

The Rust toolchain installer
https://rust-lang.github.io/rustup/
Apache License 2.0
6.17k stars 889 forks source link

rustup update fails if renaming of rust-docs fails midway #2234

Closed mdtro closed 4 years ago

mdtro commented 4 years ago

Problem My corporate system has some strict antivirus policies. When running rustup update it will usually fail at the info: installing component 'rust-docs' part because of the mass renaming of files (I assume). The security policies profile this as ransomware and begin implementing defenses against it.

Steps (this one might be hard to reproduce outside of my case)

  1. Install one of the major-brand anti-viruses (ex. Norton, McAfee, etc)
  2. Turn on the application containerization & ransomware detections
  3. Run rustup update

Possible Solution(s) My current solution is to:

  1. Let it fail the first time.
  2. Recreate the directory structure for "share/doc/rust/html"
  3. Re-run rustup update.

A more permanent solution might be to sign rustup with a valid certificate as most AV's will trust this and allow behavior that would typically be considered suspicious if the binary was not signed.

Notes System OS = Windows 10

Output of rustup --version: rustup 1.21.1 (7832b2ebe 2019-12-20) Output of rustup show:

U:\>rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  C:\Users\<redacted>\.rustup

installed targets for active toolchain
--------------------------------------

x86_64-pc-windows-msvc
x86_64-unknown-linux-gnu

active toolchain
----------------

stable-x86_64-pc-windows-msvc (default)
rustc 1.41.0 (5e1a79984 2020-01-27)
rbtcollins commented 4 years ago

Hi, we've found in the past that while we have consistent problems with anti virus it is often not the thing that people think it is and we need to drill into the specifics of the failure to identify the exact cause.

Additionally different AV products interrupt IO in different ways at different times depending on configuration, policy and so forth and this can interact with rustup as well.

Can you please provide: the actual error you are experiencing - what rustup is outputting to the console.

What specific anti-virus you are encountering this with.

It may be helpful, though I can't say if it is needed yet, to also provide a trace of the IO that rustup is going - e.g. with sysinternals process monitor or windows event tracing), configured to trace both rustup, Defender, and whatever AV and anti-malware products present on your system. I find the easiest way to do this is to filter for all file operations within ~/.rustup from any process.

mdtro commented 4 years ago

Agh sorry. I intended to include the log in my original post. I've removed some repeating lines for sake of brevity.

As for the AV products involved -- it's the McAfee enterprise suite.

I'll see what I can do about getting a dump from sysinternals or event tracing.

U:\>rustup update
info: syncing channel updates for 'stable-x86_64-pc-windows-msvc'
info: latest update on 2020-01-30, rust version 1.41.0 (5e1a79984 2020-01-27)
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: downloading component 'rust-std' for 'x86_64-unknown-linux-gnu'
info: downloading component 'rustfmt'
info: downloading component 'rls'
info: downloading component 'rust-src'
info: downloading component 'rust-analysis'
info: removing previous version of component 'rustc'
info: removing previous version of component 'rust-std'
info: removing previous version of component 'cargo'
info: removing previous version of component 'rust-docs'
info: removing previous version of component 'rust-std' for 'x86_64-unknown-linux-gnu'
info: removing previous version of component 'rustfmt'
info: removing previous version of component 'rls'
info: removing previous version of component 'rust-src'
info: removing previous version of component 'rust-analysis'
info: installing component 'rustc'
35.5 MiB / 35.5 MiB (100 %) 5.1 MiB/s in 40s ETA: 0s
info: installing component 'rust-std'
14.9 MiB / 14.9 MiB (100 %) 7.8 MiB/s in 3s ETA: 0s
info: installing component 'cargo'
info: installing component 'rust-docs'
11.9 MiB / 11.9 MiB (100 %) 252.8 KiB/s in 1m 36s ETA: 0s
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\0crcvcf20png_qvl_dir\rust-docs\share/doc/rust/html' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust/html'
[ ... ]
info: rolling back changes
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\rjrwh8r2f3ztrdun_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/x86_64-pc-windows-msvc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\rjrwh8r2f3ztrdun_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/x86_64-pc-windows-msvc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\rjrwh8r2f3ztrdun_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/x86_64-pc-windows-msvc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\rjrwh8r2f3ztrdun_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/x86_64-pc-windows-msvc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\rjrwh8r2f3ztrdun_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/x86_64-pc-windows-msvc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\rjrwh8r2f3ztrdun_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/x86_64-pc-windows-msvc'
[ ... ]
error: could not rename component file from 'C:\Users\<redacted>\.rustup\tmp\rjrwh8r2f3ztrdun_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/x86_64-pc-windows-msvc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
[ ... ]
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
error: could not rename component file from 'C:\Users\<redacted>\.rustup\tmp\lpwbg257z9ecq1sm_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
[ ... ]
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
error: could not rename component file from 'C:\Users\<redacted>\.rustup\tmp\ojcbdzm14vuoazjr_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
[ ... ]
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
error: could not rename component file from 'C:\Users\<redacted>\.rustup\tmp\u4te54n92s5f5jxk_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
[ ... ]
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
info: retrying renaming 'C:\Users\<redacted>\.rustup\tmp\7jemp9qmaojw5uwl_dir\bk' to 'C:\Users\<redacted>\.rustup\toolchains\stable-x86_64-pc-windows-msvc\share/doc/rust'
rbtcollins commented 4 years ago

Ok, so this is a duplicate of #1912 - McAfee specifically opens up handles in the directories we've written, and then those open handles prevent us moving the directory into place because of the semantics of file system locking on windows... Right now we retry a number of times, but on slower machines we time out before the scan completes.

We will see what we can do to make the situation better such as tuning the timeouts, detecting mcafee and waiting long etc. But perhaps also - I encourage you to have your corporate IT file a bug with McAfee.