Invalid lowering of macro in closure

[marmeladema]

When trying to fix https://github.com/rust-lang/rust/issues/71104 I ended up fighting with some rustdoc test about declarative macro: https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/macro-in-closure.rs

The problem is that the macro is properly lowered to hir because its exported but the parent item is not:

• Macro has def id DefId(0:5 ~ macro_in_closure[317d]::main[0]::{{closure}}[0]::m[0])
• Closure has def id DefId(0:4 ~ macro_in_closure[317d]::main[0]::{{closure}}[0])

The latter appears nowhere in the hir tree and when https://github.com/rust-lang/rust/blob/master/src/librustc_privacy/lib.rs#L945 tries to access its parent, it returns a DefId that has no associated HirId.

I don't know exactly how lowering of closure works, neither the details of declarative macro visibility, but i think the bug is either:

• the closure should have been lowered somehow
• the declarative macro should not have been lowered as it is probably not visible outside of the closure

Here is a hir dump of the test case:

warning: unused macro definition
 --> src/test/rustdoc/macro-in-closure.rs:7:9
  |
7 |         macro m() {}
  |         ^^^^^^^^^^^^
  |
  = note: `#[warn(unused_macros)]` on by default

Crate {
    item: CrateItem {
        module: Mod {
            inner: src/test/rustdoc/macro-in-closure.rs:3:1: 9:2,
            item_ids: [
                ItemId {
                    id: HirId {
                        owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
                        local_id: 0,
                    },
                },
                ItemId {
                    id: HirId {
                        owner: DefId(0:2 ~ macro_in_closure[317d]::std[0]),
                        local_id: 0,
                    },
                },
                ItemId {
                    id: HirId {
                        owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                        local_id: 0,
                    },
                },
            ],
        },
        attrs: [
            Attribute {
                kind: Normal(
                    AttrItem {
                        path: Path {
                            span: src/test/rustdoc/macro-in-closure.rs:3:4: 3:11,
                            segments: [
                                PathSegment {
                                    ident: feature#0,
                                    id: NodeId(2),
                                    args: None,
                                },
                            ],
                        },
                        args: Delimited(
                            DelimSpan {
                                open: src/test/rustdoc/macro-in-closure.rs:3:11: 3:12,
                                close: src/test/rustdoc/macro-in-closure.rs:3:22: 3:23,
                            },
                            Parenthesis,
                            TokenStream(
                                [
                                    (
                                        Token(
                                            Token {
                                                kind: Ident(
                                                    "decl_macro",
                                                    false,
                                                ),
                                                span: src/test/rustdoc/macro-in-closure.rs:3:12: 3:22,
                                            },
                                        ),
                                        NonJoint,
                                    ),
                                ],
                            ),
                        ),
                    },
                ),
                id: AttrId(0),
                style: Inner,
                span: src/test/rustdoc/macro-in-closure.rs:3:1: 3:24,
            },
        ],
        span: src/test/rustdoc/macro-in-closure.rs:3:1: 9:2,
    },
    exported_macros: [
        MacroDef {
            ident: m#0,
            vis: Spanned {
                node: Inherited,
                span: src/test/rustdoc/macro-in-closure.rs:7:9: 7:9,
            },
            attrs: [],
            hir_id: HirId {
                owner: DefId(0:5 ~ macro_in_closure[317d]::main[0]::{{closure}}[0]::m[0]),
                local_id: 0,
            },
            span: src/test/rustdoc/macro-in-closure.rs:7:9: 7:21,
            ast: MacroDef {
                body: Delimited(
                    DelimSpan {
                        open: src/test/rustdoc/macro-in-closure.rs:7:16: 7:16,
                        close: src/test/rustdoc/macro-in-closure.rs:7:21: 7:21,
                    },
                    Brace,
                    TokenStream(
                        [
                            (
                                Delimited(
                                    DelimSpan {
                                        open: src/test/rustdoc/macro-in-closure.rs:7:16: 7:17,
                                        close: src/test/rustdoc/macro-in-closure.rs:7:17: 7:18,
                                    },
                                    Paren,
                                    TokenStream(
                                        [],
                                    ),
                                ),
                                NonJoint,
                            ),
                            (
                                Token(
                                    Token {
                                        kind: FatArrow,
                                        span: src/test/rustdoc/macro-in-closure.rs:7:18: 7:19,
                                    },
                                ),
                                NonJoint,
                            ),
                            (
                                Delimited(
                                    DelimSpan {
                                        open: src/test/rustdoc/macro-in-closure.rs:7:19: 7:20,
                                        close: src/test/rustdoc/macro-in-closure.rs:7:20: 7:21,
                                    },
                                    Brace,
                                    TokenStream(
                                        [],
                                    ),
                                ),
                                NonJoint,
                            ),
                        ],
                    ),
                ),
                macro_rules: false,
            },
        },
    ],
    non_exported_macro_attrs: [],
    items: {
        HirId {
            owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
            local_id: 0,
        }: Item {
            ident: #0,
            hir_id: HirId {
                owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
                local_id: 0,
            },
            attrs: [
                Attribute {
                    kind: Normal(
                        AttrItem {
                            path: Path {
                                span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
                                segments: [
                                    PathSegment {
                                        ident: prelude_import#1,
                                        id: NodeId(4),
                                        args: None,
                                    },
                                ],
                            },
                            args: Empty,
                        },
                    ),
                    id: AttrId(2),
                    style: Outer,
                    span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
                },
            ],
            kind: Use(
                Path {
                    span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
                    res: Err,
                    segments: [
                        PathSegment {
                            ident: {{root}}#1,
                            hir_id: Some(
                                HirId {
                                    owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
                                    local_id: 1,
                                },
                            ),
                            res: Some(
                                Err,
                            ),
                            args: None,
                            infer_args: false,
                        },
                        PathSegment {
                            ident: std#1,
                            hir_id: Some(
                                HirId {
                                    owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
                                    local_id: 2,
                                },
                            ),
                            res: Some(
                                Def(
                                    Mod,
                                    DefId(1:0 ~ std[512e]),
                                ),
                            ),
                            args: None,
                            infer_args: false,
                        },
                        PathSegment {
                            ident: prelude#1,
                            hir_id: Some(
                                HirId {
                                    owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
                                    local_id: 3,
                                },
                            ),
                            res: Some(
                                Def(
                                    Mod,
                                    DefId(1:15 ~ std[512e]::prelude[0]),
                                ),
                            ),
                            args: None,
                            infer_args: false,
                        },
                        PathSegment {
                            ident: v1#1,
                            hir_id: Some(
                                HirId {
                                    owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
                                    local_id: 4,
                                },
                            ),
                            res: Some(
                                Def(
                                    Mod,
                                    DefId(1:16 ~ std[512e]::prelude[0]::v1[0]),
                                ),
                            ),
                            args: None,
                            infer_args: false,
                        },
                    ],
                },
                Glob,
            ),
            vis: Spanned {
                node: Inherited,
                span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
            },
            span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
        },
        HirId {
            owner: DefId(0:2 ~ macro_in_closure[317d]::std[0]),
            local_id: 0,
        }: Item {
            ident: std#2,
            hir_id: HirId {
                owner: DefId(0:2 ~ macro_in_closure[317d]::std[0]),
                local_id: 0,
            },
            attrs: [
                Attribute {
                    kind: Normal(
                        AttrItem {
                            path: Path {
                                span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
                                segments: [
                                    PathSegment {
                                        ident: macro_use#1,
                                        id: NodeId(10),
                                        args: None,
                                    },
                                ],
                            },
                            args: Empty,
                        },
                    ),
                    id: AttrId(1),
                    style: Outer,
                    span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
                },
            ],
            kind: ExternCrate(
                None,
            ),
            vis: Spanned {
                node: Inherited,
                span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
            },
            span: src/test/rustdoc/macro-in-closure.rs:1:1: 1:1,
        },
        HirId {
            owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
            local_id: 0,
        }: Item {
            ident: main#0,
            hir_id: HirId {
                owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                local_id: 0,
            },
            attrs: [],
            kind: Fn(
                FnSig {
                    header: FnHeader {
                        unsafety: Normal,
                        constness: NotConst,
                        asyncness: NotAsync,
                        abi: Rust,
                    },
                    decl: FnDecl {
                        inputs: [],
                        output: DefaultReturn(
                            src/test/rustdoc/macro-in-closure.rs:5:11: 5:11,
                        ),
                        c_variadic: false,
                        implicit_self: None,
                    },
                },
                Generics {
                    params: [],
                    where_clause: WhereClause {
                        predicates: [],
                        span: src/test/rustdoc/macro-in-closure.rs:5:10: 5:10,
                    },
                    span: src/test/rustdoc/macro-in-closure.rs:5:8: 5:8,
                },
                BodyId {
                    hir_id: HirId {
                        owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                        local_id: 6,
                    },
                },
            ),
            vis: Spanned {
                node: Inherited,
                span: src/test/rustdoc/macro-in-closure.rs:5:1: 5:1,
            },
            span: src/test/rustdoc/macro-in-closure.rs:5:1: 9:2,
        },
    },
    trait_items: {},
    impl_items: {},
    bodies: {
        BodyId {
            hir_id: HirId {
                owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                local_id: 2,
            },
        }: Body {
            params: [],
            value: Expr {
                hir_id: HirId {
                    owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                    local_id: 2,
                },
                kind: Block(
                    Block {
                        stmts: [],
                        expr: None,
                        hir_id: HirId {
                            owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                            local_id: 1,
                        },
                        rules: DefaultBlock,
                        span: src/test/rustdoc/macro-in-closure.rs:6:8: 8:6,
                        targeted_by_break: false,
                    },
                    None,
                ),
                attrs: ThinVec(
                    None,
                ),
                span: src/test/rustdoc/macro-in-closure.rs:6:8: 8:6,
            },
            generator_kind: None,
        },
        BodyId {
            hir_id: HirId {
                owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                local_id: 6,
            },
        }: Body {
            params: [],
            value: Expr {
                hir_id: HirId {
                    owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                    local_id: 6,
                },
                kind: Block(
                    Block {
                        stmts: [
                            Stmt {
                                hir_id: HirId {
                                    owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                                    local_id: 4,
                                },
                                kind: Semi(
                                    Expr {
                                        hir_id: HirId {
                                            owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                                            local_id: 3,
                                        },
                                        kind: Closure(
                                            Ref,
                                            FnDecl {
                                                inputs: [],
                                                output: DefaultReturn(
                                                    src/test/rustdoc/macro-in-closure.rs:6:8: 6:8,
                                                ),
                                                c_variadic: false,
                                                implicit_self: None,
                                            },
                                            BodyId {
                                                hir_id: HirId {
                                                    owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                                                    local_id: 2,
                                                },
                                            },
                                            src/test/rustdoc/macro-in-closure.rs:6:5: 6:7,
                                            None,
                                        ),
                                        attrs: ThinVec(
                                            None,
                                        ),
                                        span: src/test/rustdoc/macro-in-closure.rs:6:5: 8:6,
                                    },
                                ),
                                span: src/test/rustdoc/macro-in-closure.rs:6:5: 8:7,
                            },
                        ],
                        expr: None,
                        hir_id: HirId {
                            owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                            local_id: 5,
                        },
                        rules: DefaultBlock,
                        span: src/test/rustdoc/macro-in-closure.rs:5:11: 9:2,
                        targeted_by_break: false,
                    },
                    None,
                ),
                attrs: ThinVec(
                    None,
                ),
                span: src/test/rustdoc/macro-in-closure.rs:5:11: 9:2,
            },
            generator_kind: None,
        },
    },
    trait_impls: {},
    body_ids: [
        BodyId {
            hir_id: HirId {
                owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                local_id: 6,
            },
        },
        BodyId {
            hir_id: HirId {
                owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                local_id: 2,
            },
        },
    ],
    modules: {
        HirId {
            owner: DefId(0:0 ~ macro_in_closure[317d]),
            local_id: 0,
        }: ModuleItems {
            items: {
                HirId {
                    owner: DefId(0:1 ~ macro_in_closure[317d]::{{misc}}[0]),
                    local_id: 0,
                },
                HirId {
                    owner: DefId(0:2 ~ macro_in_closure[317d]::std[0]),
                    local_id: 0,
                },
                HirId {
                    owner: DefId(0:3 ~ macro_in_closure[317d]::main[0]),
                    local_id: 0,
                },
            },
            trait_items: {},
            impl_items: {},
        },
    },
    proc_macros: [],
warning: 1 warning emitted
}

cc https://github.com/rust-lang/rust/issues/39412

[marmeladema]

I managed to reproduce the bug with a regular macro rules:

#![feature(rustc_attrs)]

fn main() {
    || {
        #[macro_export] #[rustc_macro_transparency = "opaque"] macro_rules! m { () => {} }
    };
}

The problem is more broader that I originally thought but I still don't exactly know how to solve it quite yet.

[marmeladema]

The issue is due to an AST simplification pass: ReplaceBodyWithLoop. Before this pass, the name resolution pass creates a DefId for the macro with the closure expression as parent: fn main -> block -> closure -> block -> macrodef But later on, during ReplaceBodyWithLoop, the closure expression layer is removed and the AST ends up looking like: fn main -> block -> block -> macrodef

Then, the macro is lowered but not its parent which explains why the closure DefId does not have an associated HirId.

As far as i know, this simplification pass is only used by rustdoc.