Declarative macro support

[AlisCode]

We will need support for declarative macros, since they can output types that needs to be exported.

• We can't use cargo-expand for that, because it also expands proc macro, meaning we lose the serde attributes that we need to support.
• I didnt find any project aimed at resolving a declarative macro, only procedural macros. My understanding is that we can make one by looking at the inner implementation of macros inside the compiler and extracting that logic. That's a lot of work, including the research to relevant bits in the Rust compiler.

I'm opened to suggestions on how to support that properly.

[AlisCode]

For reference, commit bc440a810f8c90a065d43d73827562fce44828b4 has some groundwork that allows to hardcode the result of your custom macro if need be.

This needs to be documented, and more research is needed, this groundwork is mostly for quick hacking purposes

[AlisCode]

One idea to implement this (at least a version that might work, and will get us to release a version 0.1) :

On one input Rust file, we can do two pass of typebinder.

• One pass on the given codebase (so that we don't lose type annotations)
• One pass on the expanded codebase (with cargo expand or at least its internals)

That'll give us two vecs of ExportStatements, then we can dedup them, and we completely get rid of the macro solver hack that way.

[gobanos]

I may have an idea that does not require a full re-implementation of rustc macro engine (nor access to macro source :wink:)

There is a trace macro feature in nightly rustc, that can be turned on either on a scope basis, using trace_macros! or at a project basis using -Ztrace-macros=yes

The output should not be too hard to parse, even for cases where another macro call is performed at the field level: declarative macro should not hold a state and so we should be able to find the correct output in the rest of the trace.

Here are 2 demo playgrounds, both using trace_macros! but you get the same output with -Ztrace-macros=yes :

• simple case
• recursion at field level

On a side note, we can get a json output that looks like that :

{
  "reason": "compiler-message",
  "package_id": "foobar 0.1.0 (path+file:///home/gregory/Personal/foobar)",
  "manifest_path": "/home/gregory/Personal/foobar/Cargo.toml",
  "target": {
    "kind": [
      "bin"
    ],
    "crate_types": [
      "bin"
    ],
    "name": "foobar",
    "src_path": "/home/gregory/Personal/foobar/src/main.rs",
    "edition": "2018",
    "doc": true,
    "doctest": false,
    "test": true
  },
  "message": {
    "rendered": "note: trace_macro\n  --> src/main.rs:9:1\n   |\n9  | / serdify!(struct Point {\n10 | |     x: i32,\n11 | |     y: i32,\n12 | | });\n   | |___^\n   |\n   = note: expanding `serdify! { struct Point { x : i32, y : i32, } }`\n   = note: to `#[derive(Serialize, Deserialize, Debug)] struct Point { x : i32, y : i32, }`\n   = note: expanding `try! { _serde :: Serializer ::\n           serialize_struct(__serializer, \"Point\", false as usize + 1 + 1) }`\n   = note: to `match\n           _serde::Serializer::serialize_struct(__serializer, \"Point\",\n                                                false as usize + 1 + 1)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: ser :: SerializeStruct ::\n           serialize_field(& mut __serde_state, \"x\", & self.x) }`\n   = note: to `match\n           _serde::ser::SerializeStruct::serialize_field(&mut __serde_state, \"x\",\n                                                         &self.x)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: ser :: SerializeStruct ::\n           serialize_field(& mut __serde_state, \"y\", & self.y) }`\n   = note: to `match\n           _serde::ser::SerializeStruct::serialize_field(&mut __serde_state, \"y\",\n                                                         &self.y)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: de :: SeqAccess :: next_element :: < i32 > (& mut __seq) }`\n   = note: to `match _serde::de::SeqAccess::next_element::<i32>(&mut __seq)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: de :: SeqAccess :: next_element :: < i32 > (& mut __seq) }`\n   = note: to `match _serde::de::SeqAccess::next_element::<i32>(&mut __seq)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: de :: MapAccess :: next_key :: < __Field > (& mut __map) }`\n   = note: to `match _serde::de::MapAccess::next_key::<__Field>(&mut __map)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: de :: MapAccess :: next_value :: < i32 > (& mut __map) }`\n   = note: to `match _serde::de::MapAccess::next_value::<i32>(&mut __map)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: de :: MapAccess :: next_value :: < i32 > (& mut __map) }`\n   = note: to `match _serde::de::MapAccess::next_value::<i32>(&mut __map)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: de :: MapAccess :: next_value :: < _serde :: de :: IgnoredAny >\n           (& mut __map) }`\n   = note: to `match _serde::de::MapAccess::next_value::<_serde::de::IgnoredAny>(&mut __map)\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: __private :: de :: missing_field(\"x\") }`\n   = note: to `match _serde::__private::de::missing_field(\"x\")\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n   = note: expanding `try! { _serde :: __private :: de :: missing_field(\"y\") }`\n   = note: to `match _serde::__private::de::missing_field(\"y\")\n           {\n               _serde :: __private :: Ok(__val) => __val, _serde :: __private ::\n               Err(__err) => { return _serde :: __private :: Err(__err) ; }\n           }`\n\n",
    "children": [
      {
        "children": [],
        "code": null,
        "level": "note",
        "message": "expanding `serdify! { struct Point { x : i32, y : i32, } }`",
        "rendered": null,
        "spans": []
      },
      {
        "children": [],
        "code": null,
        "level": "note",
        "message": "to `#[derive(Serialize, Deserialize, Debug)] struct Point { x : i32, y : i32, }`",
        "rendered": null,
        "spans": []
      },
      /* <serde stuff> */
    ],
    "code": null,
    "level": "note",
    "message": "trace_macro",
    "spans": [
      {
        "byte_end": 216,
        "byte_start": 165,
        "column_end": 4,
        "column_start": 1,
        "expansion": null,
        "file_name": "src/main.rs",
        "is_primary": true,
        "label": null,
        "line_end": 12,
        "line_start": 9,
        "suggested_replacement": null,
        "suggestion_applicability": null,
        "text": [
          {
            "highlight_end": 24,
            "highlight_start": 1,
            "text": "serdify!(struct Point {"
          },
          {
            "highlight_end": 12,
            "highlight_start": 1,
            "text": "    x: i32,"
          },
          {
            "highlight_end": 12,
            "highlight_start": 1,
            "text": "    y: i32,"
          },
          {
            "highlight_end": 4,
            "highlight_start": 1,
            "text": "});"
          }
        ]
      }
    ]
  }
}