Having rust-analyzer open causes linker errors for pyo3

mejrs commented 3 years ago

When compiling pyo3 having rust-analyzer open causes the build to fail.

I can reliably reproduce this by just opening vscode, letting rust analyzer run whatever it does on start, and then trying cargo test in another terminal. Closing vscode or disabling rust-analyzer fixes this issue.

Error message

``` PS C:\Users\bruno\rust\pyo3> cargo test Compiling pyo3-build-config v0.14.2 (C:\Users\bruno\rust\pyo3\pyo3-build-config) Compiling pyo3-macros-backend v0.14.2 (C:\Users\bruno\rust\pyo3\pyo3-macros-backend) Compiling pyo3 v0.14.2 (C:\Users\bruno\rust\pyo3) Compiling pyo3-macros v0.14.2 (C:\Users\bruno\rust\pyo3\pyo3-macros) error: linking with `link.exe` failed: exit code: 1201 | = note: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.25.28610\\bin\\HostX64\\x64\\link.exe" "/DEF:C:\\Users\\bruno\\AppData\\Local\\Temp\\rustcyt6mCk\\lib.def" "/NOLOGO" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.10w5m3jfg09mphew.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.12atyo2j85fkq7cn.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.18483pet6oorfcaf.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.1b3gl59olbl078rv.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.1m516po019l0b1r0.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.1wyuuzmii0z3mba5.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.22w8l3o2n1cmns6e.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.267j86se45yb1fvv.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.28wp4el970y0e33h.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2bavf68ka8l7oan1.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2bex68l6ix2kiofz.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2gi9orfha1rolc6d.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2k56a8e75mrlw5yq.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2qmuwrycwz70sd3r.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2qzdjvbv7hajpo6c.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2r0fy647j9b3hqtk.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.2t34msl2e3ek8yj3.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.323b3zme49dll2dm.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.32d21jzu6bo79or2.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.32smgizh9mxo6roh.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.32t8a1mf2ij8lkcc.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.37df7aase994kf15.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.3gnt06ozs9hmjt2d.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.3oruw5wrjrsvh5ec.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.3tmz0o8r8aco6sqy.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.43z145u4nglohhs4.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.46wbd82zqw72aj7e.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.4g9xnpwh2vjmi7jf.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.4xumjuzdihsg8ooi.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.514kzbey4lomdmw2.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.514m8ci3mrny103g.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.56aothl0zz1nzas0.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.57dal3gkx80gusjc.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.5y99v0txi0lz2gl.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.9h80lnnnvctvzip.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.f3le6jgmclil3yz.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.pw3alhz0cjjtz8j.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.zqpuu5wupl9fsb5.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.3plen2l1azk7e7wv.rcgu.o" "C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.26n76gurlhdmqadw.rcgu.o" "/LIBPATH:C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps" "/LIBPATH:C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\bruno\\Rust\\pyo3\\target\\debug\\deps\\libpyo3_macros_backend-abe1702ae06f2fcc.rlib" "C:\\Users\\bruno\\Rust\\pyo3\\target\\debug\\deps\\libpyo3_build_config-7e1685a275eab5ab.rlib" "C:\\Users\\bruno\\Rust\\pyo3\\target\\debug\\deps\\libonce_cell-9a97b8357599c0b9.rlib" "C:\\Users\\bruno\\Rust\\pyo3\\target\\debug\\deps\\libsyn-e55518d63704e3dd.rlib" "C:\\Users\\bruno\\Rust\\pyo3\\target\\debug\\deps\\libquote-8e69fd11733a9ec9.rlib" "C:\\Users\\bruno\\Rust\\pyo3\\target\\debug\\deps\\libproc_macro2-8ce88383efc71c4a.rlib" "C:\\Users\\bruno\\Rust\\pyo3\\target\\debug\\deps\\libunicode_xid-93f2aaaeab33b3e4.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libproc_macro-cf4b0cc3ea285d94.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-stlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-65e73515f6680323.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd_detect-f8cff8d9bd7840ad.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-fe60ba8a15823ad7.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-cf86dd7881d3c2f9.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-43c65575cdb9db58.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-b6defdcc73190cb\\x86_64-pc-windows-msvc\\lib\\liblibc-29274e913e0d2be1.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-628c6d389039a057.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-5253a24e114dde59.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-230e4a83f184a187.rlib" "C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-d4faa97442f81921.rlib" "kernel32.lib" "ws2_32.lib" "advapi32.lib" "userenv.lib" "kernel32.lib" "msvcrt.lib" "/NXCOMPAT" "/LIBPATH:C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "/OUT:C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.dll" "/OPT:REF,NOICF" "/DLL" "/IMPLIB:C:\\Users\\bruno\\rust\\pyo3\\target\\debug\\deps\\pyo3_macros-d7af52fdb111651f.dll.lib" "/DEBUG" "/NATVIS:C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\bruno\\.rustup\\toolchains\\nightly-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" = note: LINK : fatal error LNK1201: error writing to program database 'C:\Users\bruno\rust\pyo3\target\debug\deps\pyo3_macros-d7af52fdb111651f.pdb'; check for insufficient disk space, invalid path, or insufficient privilege error: could not compile `pyo3-macros` due to previous error PS C:\Users\bruno\rust\pyo3> cargo clean error: could not remove build directory Caused by: failed to remove file `C:\Users\bruno\rust\pyo3\target\debug\deps\pyo3_macros-d7af52fdb111651f.pdb` Caused by: Access is denied. (os error 5) ```

lnicola commented 3 years ago

We don't do much more than running cargo check, and cargo should use a lock to avoid problems when you run cargo test outside of RA.

check for insufficient disk space, invalid path, or insufficient privilege

I guess that's not the problem, though.

mejrs commented 3 years ago

I think this might have to do with running nightly rust - I tried some more things but can't reproduce it on stable.

rust-analyzer-x86-64-pc-windows-msvc.exe is holding a file lock on that .pdb file. Killing that process fixes this also.

lnicola commented 3 years ago

Hmm. We make a copy of the proc macro DLL and load that, so we avoid stepping on cargo's toes. Does it still point to the PDB? I don't really remember.

bjorn3 commented 3 years ago

According to https://docs.microsoft.com/en-us/visualstudio/debugger/specify-symbol-dot-pdb-and-source-files-in-the-visual-studio-debugger?view=vs-2019 the linker by default embeds the full path to the pdb file in the exe file.

lnicola commented 3 years ago

But still, does the loader really lock the PDB? Or was that a red herring?

asv7c2 commented 2 years ago

It affected me too. Using stable MSVC toolchain. It mostly happens when frequently modify proc macro during developing. Restaring VSC or RA only helping. Linker just cannot write to proc macro's pdb file.

stanciuadrian commented 2 years ago

It works on my machine ™️: Windows 11 x64 & VS2022 toolchain.

@mejrs could you please run the same commands and send us the output? listdlls and handle64 belong to Sysinternals Suite. The PID (26420) is the rust-analyzer process with lower memory footprint (private bytes). You can use Task Manager or procexp64.

> cargo test
test result: ok. 341 passed; 0 failed; 47 ignored; 0 measured; 0 filtered out; finished in 36.70s

> rust-analyzer.exe --version
rust-analyzer 24cf95762 2022-04-11 stable

> rustup update
info: syncing channel updates for 'stable-x86_64-pc-windows-msvc'
info: syncing channel updates for 'nightly-x86_64-pc-windows-msvc'
info: checking for self-updates

   stable-x86_64-pc-windows-msvc unchanged - rustc 1.60.0 (7737e0b5c 2022-04-04)
  nightly-x86_64-pc-windows-msvc unchanged - rustc 1.62.0-nightly (1f7fb6413 2022-04-10)

info: cleaning up downloads & tmp directories

> listdlls 26420
Listdlls v3.2 - Listdlls
Copyright (C) 1997-2016 Mark Russinovich
Sysinternals

------------------------------------------------------------------------------
rust-analyzer.exe pid: 26420
Command line: "c:\Users\\.vscode\extensions\matklad.rust-analyzer-0.2.1010-win32-x64\server\rust-analyzer.exe" proc-macro

Base                Size      Path
0x0000000016060000  0x1975000  c:\Users\\.vscode\extensions\matklad.rust-analyzer-0.2.1010-win32-x64\server\rust-analyzer.exe
0x000000006efc0000  0x209000  C:\WINDOWS\SYSTEM32\ntdll.dll
0x000000006e600000  0xbd000   C:\WINDOWS\System32\KERNEL32.DLL
0x000000006cae0000  0x374000  C:\WINDOWS\System32\KERNELBASE.dll
0x000000006cf40000  0x7ae000  C:\WINDOWS\System32\shell32.dll
0x000000006c7c0000  0x9d000   C:\WINDOWS\System32\msvcp_win.dll
0x000000006c6a0000  0x111000  C:\WINDOWS\System32\ucrtbase.dll
0x000000006e890000  0x1ac000  C:\WINDOWS\System32\USER32.dll
0x000000006cab0000  0x26000   C:\WINDOWS\System32\win32u.dll
0x000000006cf10000  0x29000   C:\WINDOWS\System32\GDI32.dll
0x000000006c8d0000  0x112000  C:\WINDOWS\System32\gdi32full.dll
0x000000006df80000  0x8000    C:\WINDOWS\System32\PSAPI.DLL
0x000000006be90000  0x27000   C:\WINDOWS\SYSTEM32\bcrypt.dll
0x000000005fcf0000  0x1b000   C:\WINDOWS\SYSTEM32\VCRUNTIME140.dll
0x000000006e030000  0x31000   C:\WINDOWS\System32\IMM32.DLL
0x000000006c4b0000  0x7f000   C:\WINDOWS\System32\bcryptprimitives.dll
0x000000006ce60000  0xae000   C:\WINDOWS\System32\advapi32.dll
0x000000006de50000  0xa3000   C:\WINDOWS\System32\msvcrt.dll
0x000000006df90000  0x9e000   C:\WINDOWS\System32\sechost.dll
0x000000006ebd0000  0x120000  C:\WINDOWS\System32\RPCRT4.dll
0x0000000069e20000  0x34000   C:\WINDOWS\SYSTEM32\ntmarta.dll
0x000000002d2c0000  0x5f000   C:\Users\\AppData\Local\Temp\16578131557812546816indoc-f4c9a4dbd503caad.dll
0x000000000f9a0000  0x91000   C:\Users\\AppData\Local\Temp\5949480503956475169proc_macro_error_attr-c561d03c93d0e8bb.dll
0x00000000f5460000  0x3fa000  C:\Users\\AppData\Local\Temp\2161299095135425567pyo3_macros-f9f2db4b2516e123.dll
0x0000000003190000  0x76000   C:\Users\\AppData\Local\Temp\3142464688797423265rustversion-a3785f1caada41da.dll
0x00000000ef780000  0x3be000  C:\Users\\AppData\Local\Temp\14672087416760959905serde_derive-b1699c950b45c143.dll
0x0000000051590000  0x18000   C:\ProgramData\Microsoft\Windows Defender\Platform\4.18.2203.5-0\MpDetoursCopyAccelerator.dll
0x000000006e5a0000  0x5d000   C:\WINDOWS\System32\SHLWAPI.dll
0x00000000fa2f0000  0x2ee000  C:\Users\\AppData\Local\Temp\13785480067089490867structopt_derive-d47d9012ce1a23a7.dll

> handle64 -p 26420
Nthandle v4.22 - Handle viewer
Copyright (C) 1997-2019 Mark Russinovich
Sysinternals - www.sysinternals.com

  1B4: File  (RW-)   C:\Rust\pyo3
  2A0: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\exthost1\output_logging_20220411T211832\4-Rust Analyzer Client.log
  3B4: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\exthost1\exthost.log
  3F8: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\exthost1\output_logging_20220411T211832\1-Microsoft Authentication.log
  3FC: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\exthost1\output_logging_20220411T211832\6-GitHub Authentication.log
  418: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\exthost1\output_logging_20220411T211832\2-EditorConfig.log
  420: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\exthost1\output_logging_20220411T211832\5-Debug.log
  43C: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\exthost1\output_logging_20220411T211832\3-Git.log
  6C8: Section       \Sessions\1\BaseNamedObjects\C:*ProgramData*Microsoft*Windows*Caches*{6AF0698E-D558-4F6E-9B3C-3716689AF493}.2.ver0x0000000000000009.db
  8C8: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\main.log
  A4C: File  (RW-)   C:\Users\\AppData\Roaming\Code\logs\20220411T211830\renderer1.log
  A60: Section       \Sessions\1\BaseNamedObjects\SessionImmersiveColorPreference
  A68: Section       \Sessions\1\BaseNamedObjects\C:*ProgramData*Microsoft*Windows*Caches*cversions.2.ro
  AB8: Section       \Sessions\1\BaseNamedObjects\C:*ProgramData*Microsoft*Windows*Caches*cversions.2.ro
  AC4: Section       \Sessions\1\BaseNamedObjects\C:*ProgramData*Microsoft*Windows*Caches*{DDF571F2-BE98-426D-8288-1A9A39C3FDA2}.2.ver0x0000000000000001.db

andylizi commented 2 years ago

This is caused by rust-lang/backtrace-rs#470. It happens when a proc macro panics and prints a backtrace. There's probably no way to fix it from our side.

A workaround is to disable backtrace by setting the RUST_BACKTRACE env variable:

    "rust-analyzer.server.extraEnv": {
        "RUST_BACKTRACE": "0"
    },

rust-lang / rust-analyzer

Having rust-analyzer open causes linker errors for pyo3 #9932