Allow access to location information from proc_macro TokenStreams

[djc]

I would like to build a template engine that leverages custom derive to attach a template rendering function to a struct at compile time. I was planning on attaching an attribute to the struct to specify the path to the template file, which could then be read and transpiled into Rust. However, it seems it is currently not possible to get at the path to the source file where the TokenStream originated. As such, it seems I would have to rely on absolute paths to template files or on the current working directory. Even assuming cargo compiles (including workspaces), this does not seem reliable.

The Items that are part of the TokenStream contain a Span, but I don't think the CodeMap they index into is currently available to a proc_macro. Would that be a viable implementation strategy?

[djc]

My experimental code can be found here: https://github.com/djc/askama

[LukasKalbertodt]

Is there any update on this? I need this feature, too: I'm writing a macro that has its own "module system" so to speak. Thus I need to load files form disk relative to the file in which the macro is called.

@jseyfried @alexcrichton I could try implementing this feature for proc_macro. What do you think? Are there reasons not to expose the current path to the macro author?

[alexcrichton]

There's some work in https://github.com/rust-lang/rust/pull/43604 but AFAIK there's no movement towards manufacturing custom spans. I know it's something we'd like to support eventually!

[abonander]

43604 is mainly just waiting on review right now.

[jyn514]

This is Span::call_site, right? That's stable: https://doc.rust-lang.org/stable/proc_macro/struct.Span.html#method.call_site

[djc]

I have a hard time remembering exactly my intent with this issue from 5 years ago, but I think this would depend on getting proc_macro_span stabilized (for the Span::source_file() method).

[abonander]

What would actually be more useful, less work to stabilize and have additional utility, is to basically just be able to resolve a file path the way include!() or include_str!() does, given a span or SourceFile.

Then like the new proc_macro_tracked_path feature, the compiler could know to watch that file for changes and trigger recompilation.

@petrochenkov I feel like it would actually make a lot of sense to just combine these. Maybe something like:

impl SourceFile {
    /// Load a file relative to this source file and parse it as a `TokenStream`.
    ///
    /// The file path will be included in the resulting dep-info and will trigger recompilation when it changes.
    pub fn include(&self, path: impl AsRef<Path>) -> Result<TokenStream, IncludeError> { ... }

    /// Load a file relative to this source file as a UTF-8 string.
    ///
    /// The file path will be included in the resulting dep-info and will trigger recompilation when it changes.
    pub fn include_str(path: impl AsRef<Path>) -> Result<String, IncludeError> { ... }
}

This would actually fix a long-outstanding issue in SQLx (although I think include_str!() would fix it, just haven't gotten around to trying it): https://github.com/launchbadge/sqlx/issues/663#issuecomment-683476687

[djc]

Yeah, that would be pretty great to have for my use case.