Tracking issue for RFC 2091: Implicit caller location

[aturon]

This is a tracking issue for the RFC "Implicit caller location" (rust-lang/rfcs#2091).

Steps:

• [x] Implement the RFC (cc @rust-lang/compiler -- can anyone write up mentoring instructions?)
• [ ] Adjust documentation (see instructions on forge)
• [x] Stabilization PR (see instructions on forge)

Unresolved questions:

• ~~If we want to support adding #[track_caller] to trait methods, the redirection pass/query/whatever should be placed after monomorphization, not before. Currently the RFC simply prohibits applying #[track_caller] to trait methods as a future-proofing measure.~~

• Diverging functions should be supported.

• ~~The closure foo::{{closure}} should inherit most attributes applied to the function foo, in particular #[inline], #[cold], #[naked] and also the ABI. Currently a procedural macro won't see any of these, nor would there be anyway to apply these attributes to a closure. Therefore, #[rustc_implicit_caller_location] currently will reject #[naked] and ABI, and leaving #[inline] and #[cold] mean no-op. There is no semantic reason why these cannot be used though.~~

[eddyb]

@petrochenkov There is 0 MIR inlining going on in my example, sorry for the confusion (@Aaron1011 brought up in regards to MIR inlining but the problem is always codegen of a function defined in another crate). MIR inlining isn't even on by default yet.

The problem I noted is entirely because we can't get the invocation span (because the ExpnId is gone cross-crate), so the panic::Location refers to inside the macro instead of the invocation site of the macro.

[est31]

cc #69977

[anp]

Checking in and it looks like #68941 has merged, @Aaron1011 is there a PR I can follow for the MIR-serialized-hygiene-info?

[Aaron1011]

@anp: The next step is https://github.com/rust-lang/rust/pull/70077, which is currently in the merge queue. I plan to open an WIP PR for hygiene serialization fairly soon.

[anp]

A question came up in code review re: the attribute's behavior in traits with specialization, I filed an issue for follow-up since specialization is unstable and no currently planned uses of the attribute require an answer.

[anp]

https://github.com/rust-lang/rust/pull/69251 is in the merge queue, implementing the attribute in traits.

[anp]

Support for the attribute in traits has landed, currently working on landing Index(Mut) support in #70234.

[anp]

I finished the first full draft of the section of the rustc-dev-guide documenting the feature, the PR is still open. I'm sure there are aspects of this that aren't covered sufficiently or additional material that could be helpful. Feedback on the PR very much appreciated!

I think that the only remaining implementation question is how to handle -Zlocation-detail-control={file,line,column} from the RFC. I'm not sure how useful the multiple levels of granularity are.

It seems like there are two motivations that have come up for the option:

1. to preserve pre-existing optimization opportunities in cold error-handling codepaths
2. to mitigate binary size bloat from additional file paths being included in the binary (which aren't stripped w/ debug info)

(1) was a concern at the time of the RFC but we've not yet seen any performance regressions to suggest that this is happening to the serious detriment of any projects (projects that are very sensitive to code size of panicking already avoid a lot of that code at all costs).

(2) seems like a legitimate concern and could be addressed with a simpler -Zredact-caller-location flag to make all Locations the same. Incidentally this might allow for even more opportunities in (1).

IIUC in order to implement this we need to preserve the types/layouts/etc of all of the APIs since e.g. std is precompiled for most builds and the binaries must be compatible. Is it as simple as changing the allocation functions to replace filenames with <redacted> and setting the line/column numbers to 0?

[eddyb]

I was suggesting elsewhere that we could be using sections. Specifically, two:

• the string data for filenames would go into one section, which could be discarded
• the Locations i.e. (&str, u32, u32) would go into another section, which would be zeroed
  • Location will have to use Option<&str> for filenames, instead of &str, for this to be sound

However, I'm not sure there's a good way to automate that treatment, and it'd be a bit excessive for us to make our tools for this. But we do control the last linker invocation, so we could give it instructions?

Is it as simple as changing the allocation functions to replace filenames with and setting the line/column numbers to 0?

For RUSTFLAGS=-Zredact-caller-location that can't hide Locations already compiled into core/alloc/std, sure, and I suppose for many purposes that might be enough.

Should we have a single flag that also affects file!()/line!()/column!() too? What about debug info?

[Centril]

(2) seems like a legitimate concern and could be addressed with a simpler -Zredact-caller-location flag to make all Locations the same. Incidentally this might allow for even more opportunities in (1).

This seems like a reasonable compromise. 👍

[anp]

Update to the binary size investigation: a custom section shows some measurable size usage but I'm unsure whether it's enough to gate stabilization on a mitigation strategy. Input on the measurement issue appreciated!

[anp]

EDIT: moving the drafted stabilization report to the stabilization PR

[scottjmaddox]

Is there a reason file(), column(), and line() aren't const fns?

Also, is there a reason for not having module_path()?

[nikomatsakis]

Bug that @eddyb raised relating to panics and cross-crate macro spans:

https://github.com/rust-lang/rust/issues/70963

[anp]

I've updated the stabilization report with a couple more bug links, a better opening, and a history of PRs for the feature.

[nikomatsakis]

@anp one thing I feel like I am missing is what the "scope" of this stabilization report covers. I'd like a kind of checklist of the new things that are possible now if we opt to stabilize.

I believe it is:

• track_caller can be placed on functions in all locations, and it will influence the results of the Location APIs and the output from panics and the like

but I am not sure whether the Location APIs themselves are, for example, covered under this stabilization.

I think what we would normally do, also, is to create an issue dedicated to the stabilization, or perhaps a PR that actually makes the changes. That's a bit cleaner than using the tracking issue.

[anp]

Good idea! I’ll work on that.

[joshtriplett]

I'm going to temporarily un-nominate this given the request from @nikomatsakis, but please feel free to re-nominate this once updated with that information.

[ggriffiniii]

I scanned the history but didn't see this question mentioned. Has there been any discussion about moving the Location type out of the std::panic module, or even just re-exporting under a different name if it needs to stay there for backwards compatibility?

I'm really looking forward to track_caller, but almost none of my use cases are panic related and panic is a pretty eye-catching word in code that causes me to do a double-take every time I encounter it. It seems like a superficial thing to discuss compared with all the implementation details, but I just wanted to provide my experience in case that aspect has been overlooked.

[anp]

This has come to mind for me a few times and I’m glad you brought it up. Seems worth filing an issue for discussion?

[est31]

What would be non-panic-related use cases of this feature? It only affects panic information.

As for moving the location out of the std::panic module, there are three ways the standard library crates (including proc_macro) display location information. First is through std::panic::Location, second is std::backtrace::Backtrace, third is proc_macro::Span. They use three distinct methods of passing the data specific to their way of functioning. If Location is moved out of std::panic, care should be taken to avoid confusion with the other methods.

[jethrogb]

It only affects panic information.

Not anymore, you can just call Location::caller() anywhere and use it, independently of panicking/unwinding.

[dtolnay]

Labeling T-libs because std::panic::Location::caller is covered by this tracking issue.

[est31]

@jethrogb good point, thanks.

[anp]

Alright! I've revised the PR to the reference and have opened a PR to stabilize the attribute and wrapper: https://github.com/rust-lang/rust/pull/72445.

[ivnsch]

Sorry, what is the name of the flag? location-detail and redact-caller-location are returning "unknown flag"

[anp]

The flag needs implementation still: https://github.com/rust-lang/rust/issues/70580.

[anp]

The stabilization PR just landed! The PR to the reference should be ready to go as well, after which I think we can close this tracking issue (at least those are the only checkboxes remaining in the top message).

[nikomatsakis]

Amazing work, @anp!

[elichai]

This is awesome! One problem I have is that it's not possible to put #[track_caller] on closures, this is really useful when thread_local is involved because then there's a lot of logic in closures that you might want to propagate ie

thread_local! {static DONE: bool = false;}

#[track_caller]
fn assert_done() {
    DONE.with(
        #[track_caller]
        |b| assert!(b),
    );
}

fn main() {
    assert_done();
}

[anp]

The restriction on closures were in the original RFC, I think as a result of the implementation proposed then. Right now, I can't think of a reason the current implementation couldn't support this but I could easily be missing something. I opened https://github.com/rust-lang/rust/issues/74042 to discuss/track.

[xd009642]

Apologies if this has been raised before, but I've been playing around with trying to track where allocations happen with something like so:

use libc_print::libc_println;
use std::alloc::{GlobalAlloc, Layout};
use std::panic::Location;

pub struct TracedAlloc<T: GlobalAlloc> {
    pub allocator: T,
}

unsafe impl<T> GlobalAlloc for TracedAlloc<T>
where
    T: GlobalAlloc,
{
    #[track_caller]
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        libc_println!("Alloc {:?} at {:?}", layout, Location::caller());
        self.allocator.alloc(layout)
    }

    #[track_caller]
    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
        libc_println!("Dealloc {:?} at {:?}", layout, Location::caller());
        self.allocator.dealloc(ptr, layout)
    }
}

However the caller location is always the line I've put #[global_allocator] which makes #[track_caller] useless in this context.

[KodrAus]

Are we happy to close this tracking issue now that #74042 has spun off work to support the attribute on closures?

[nikomatsakis]

Sure, I guess so. Thanks again @anp for seeing this through!

[schreter]

Hi. I know this is a bit off-topic and this particular issue is already closed, but I suppose, the right people are involved here :-), so someone might help.

I implemented for our project a version of Result which uses Try trait in such a way as to track the error handling locations, so we get a "call stack" of the error at the end. This works in conjunction with #[track_caller] very well and we see the locations the error handling took.

However, there is one major deficiency in Location - it only gives us the source name and line. Yes, with that, it's possible to manually look up the function where it happened, but it would be significantly simpler to evaluate bug reports by looking at the call trace with function names (we have practiced this in a large C++ project, where basically the dev support standardized on initially evaluating everything by function names call trace, ignoring line numbers). So it would come extremely handy if the Location could be extended with another &str containing the function name (ideally with generics). It can be also a mangled name, I don't care much, but the function name is important.

Before you should "backtrace!" - yes, but... We are using heavy asynchronous processing handling errors across awaits, where the backtrace has about zero value. Similar for tracing, we can't just always trace due to performance reasons. So the error handling "call stack" is a perfect compromise - cheap and sufficiently valuable (except that it's missing the function name).

Any suggestion where/how to address this issue (Location extension by file name)? According to my code study of the Rust compiler code, it should basically boil down to getting the function name from the current Span and adding it in addition to the file name to the generated const Location here: https://github.com/rust-lang/rust/blob/012720ffb075a087b781325d17d1822a340a2f2a/compiler/rustc_codegen_cranelift/src/common.rs#L351

Thanks & regards,

Ivan

[jplatte]

I remember asking about this quite a while ago and getting a response along the lines of "unlikely to happen for perf or code size reasons". Unfortunately I can't find an issue about it, so I guess it must have been on Zulip.

I think creating a separate issue would be more useful than continuing this conversation here, in any case.

[schreter]

I remember asking about this quite a while ago and getting a response along the lines of "unlikely to happen for perf or code size reasons". Unfortunately I can't find an issue about it, so I guess it must have been on Zulip.

Well, it's clear that this will increase generated constants segment by potentially quite a bit, because instead of a single string per file we'll need to store many strings per file. So it could be made optional at compilation time. But I personally don't think it'll be significantly slower (during compilation; runtime is obviously unaffected).

I think creating a separate issue would be more useful than continuing this conversation here, in any case.

Sure. Simply a new top-level issue w/o any special tags? The question is how to get it to the attention of relevant people who could do something about it.

[est31]

If you have the source code, it should be quite easy to build a tool that translates the Location to the name of the function. Location contains the file name, as well as line and column, so all you need to do is go to that specific line and column, and then parse the next ident. Done easily with syn where you can write a custom visitor with a visit_ident function, like this. Then you just compare the line/col number with the line/col number of the Location, done. Maybe if the source code is not public you can export the Location via e.g. json and import it via the tool.

[schreter]

If you have the source code, it should be quite easy to build a tool that translates the Location to the name of the function.

Of course it could be translated. But that's another step, which makes it quite cumbersome. The compiler already knows it at compile time and aside from costing more space in the generated executable (string section), there should be no adverse effects of having the function name in the Location. It would in general allow building various tools which use the location for debugging purposes, which in turn would help the community in general.

Let me move this to a new issue, for further discussion.