search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.53k stars 12.74k forks source link

Proc Macro Crate and Workspace Miri and Clippy Malloc Error #92185

Open KSBilodeau opened 2 years ago

KSBilodeau commented 2 years ago

When compiling a workspace that has a proc macro crate in it, rustc emits a malloc error causing tools such as miri and clippy to fail. This bug affects nightly versions 2021-12-05+.

I tried this code:

playground_workspace.zip

I expected to see this happen: Miri and clippy successfully assess my code

Instead, this happened: the following crashes (I can supply the actual Mac IPS files if it would be more helpful than these txt copies): clippy.txt miri.txt

Meta

rustc --version --verbose:

rustc 1.59.0-nightly (23f69235a 2021-12-20)
binary: rustc
commit-hash: 23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a
commit-date: 2021-12-20
host: x86_64-apple-darwin
release: 1.59.0-nightly
LLVM version: 13.0.0
hkratz commented 2 years ago

Looks like it is during proc macro compilation, not execution? Shouldn't that use jemalloc instead of the system malloc?

The error message is the same as in #92173.

ehuss commented 2 years ago

clippy and miri are not built with jemalloc.

I think this is likely the same as #92173.

I've been trying to dig into this some. I bisected to #91505, and I've been trying to bisect within that rollup, but I'm not really finding out much. I'm still trying to go through them, it is a pretty slow process.

ehuss commented 2 years ago

I'm a bit at a loss. Reverting #91385 makes the problem go away (both here and my tests with #92173). However, that PR is not doing anything that should affect any of this, since it is only within an error path. So my best guess is that PR shuffles the code layout/generation just enough to trigger the underlying issue.

cc @rust-lang/clippy and @rust-lang/miri just as a heads up that clippy and miri are crashing on macOS. I haven't tested RLS, but I would imagine it might also have issues.

I would be reluctant to enable jemalloc in miri and clippy. I don't know what is causing the underlying error. However, that is a route we could take. It would involve replicating all of this in their respective main binaries. I'm having dejavu about this, but I don't remember where I last discussed this.

RalfJung commented 2 years ago

I assume this is the same problem as https://github.com/rust-lang/miri/issues/1941

RalfJung commented 2 years ago

I'm a bit at a loss. Reverting #91385 makes the problem go away

At least that is consistent with the regression range from rust-lang/miri#1941.

Miri itself acts as rustc for proc-macros, but it seems (I was not aware of that) it is a different rustc than the "real" one -- it does not use jemalloc? So yeah I assume this is the same as https://github.com/rust-lang/rust/issues/92173. Though I also assume that just masks the error since jemalloc doesn't do the realloc check that leads to the abort. That's my guess anyway.

rdornin commented 2 years ago

I just want to report that issue is currently blocking me in development. If there are any work around for the macos darwin tool chain and clippy errors... would love to hear about it.