alexcrichton commented 6 years ago

This issue is intended to track a number of unstable APIs which are used to inspect the contents of a Span for information like the file name, byte position, manufacturing new spans, combining them, etc.

This issue tracks the proc_macro_span unstable feature.

Public API

Already stabilized:

impl Span {
    pub fn source_text(&self) -> Option<String>;
}

impl Group {
    pub fn span_open(&self) -> Span;
    pub fn span_close(&self) -> Span;
}

To be stabilized, probably in their current form:

impl Span {
    pub fn line(&self) -> usize;
    pub fn column(&self) -> usize;

    pub fn start(&self) -> Span;
    pub fn end(&self) -> Span;
}

To be stabilized after some (re)design or discussion:

impl Span {
    pub fn source_file(&self) -> SourceFile;

    pub fn byte_range(&self) -> Range<usize>;
}

#[derive(Clone, Debug, PartialEq, Eq)]
pub struct SourceFile { .. }

impl !Send for SourceFile {}
impl !Sync for SourceFile {}

impl SourceFile {
    pub fn path(&self) -> PathBuf;
    pub fn is_real(&self) -> bool;
}

Things that require more discussion:

impl Span {
    pub fn eq(&self, other: &Span) -> bool;
    pub fn join(&self, other: Span) -> Option<Span>;
    pub fn parent(&self) -> Option<Span>;
    pub fn source(&self) -> Span;
}

impl Literal {
    pub fn subspan<R: RangeBounds<usize>>(&self, range: R) -> Option<Span>;
}

softprops commented 5 years ago

I wanted to shed some pain this is causing with tarpaulin.

Tarpaulin has worked amazingly well for me and my company as a replacement for kcov which historically been a pain to get accurate, reliable and correct results. At the moment tarpaulin stands as the most promising goto option for codecoverage in rust and just feels more like the only first class tooling option.

Having one of those when choosing to invest in a technology is important for many company's adoption story for checking off code quantity checkmarks. When they see that rust doesn't have a reasonable code quality story that works on stable rust, that's can result in a "pass" rather than "I'll take it". There are currently some work arounds for making this work-ish on stable but it feels very much like the story serde was in a year or so ago when you wanted to show all your friends how amazing serde was but then was embarrassed to show what it took to make work on stable because of a macro stabilization blocker.

JakeTherston commented 5 years ago

With procedural macros having reached a point where they're very useful on stable, I expect many users will find themselves needing access to this information. Would it be reasonable to only stabilize parts of the Span API that are not too risky? Perhaps exposing a function that optionally returns the path of the file where a macro is invoked if such a file exists?

matklad commented 5 years ago

I have a concern about exposing LineColum information. It looks like it could play badly with incremental compilation, especially in the IDE context.

My understanding is that, if one adds a blank line to the start of the file, the line_column information of the input spans to proc macro changes. That means that IDE would have to re-expand procedural macros even after insignificant white space changes.

I would feel more confident if proc-macros were strictly a pure function from the input token stream to output token stream. This can be achieved, for example, by making line-column infocation relative to the start of macro invocation (as opposed to relative to the start of the file).

I don't feel that exposing absolute position is necessary the end of the world: IDE can track if a macro actually makes use of the info, or it can supply macro with some fake line/columns. But it's hard to tell if such hacks would work well in practice, and it's hard to experiment with them for the lack of IDE which handled proc-macros incrementally....

davidlattimore commented 5 years ago

If the parser allocated IDs to every AST node, then, and this is the hard part, when an edit was made to the source, the parser tried to keep those IDs the same in all the non-edited code and only allocate new IDs for new nodes, that would allow spans to be kept completely separate from the AST. Those IDs could be passed through macro expansion without causing unnecessary invalidations. If something needed a span later on, it could then go back and ask the parser for the span for that particular AST node ID. I feel like having an incremental parser is important, not because parsing is the bottleneck, but because it underpins everything else.

matklad commented 5 years ago

@davidlattimore this is fascinating, but slightly off-topic for the issue. I've created a thread on internals: https://internals.rust-lang.org/t/macros-vs-incremental-parsing/9323