Cannot register macro with same name

[jakobjpeters]

The current design limits the extensibility of the package, and I argue that no macro should be given special treatment. While I'm not sure if this is an issue when a user depends on a package registering the same name, the predefined behavior intended for Distributed.@everywhere, Turing.@model, and MacroTools.@capture prevents anyone else from registering their own macro under one of those names. This applies to the Base and Core macros too, but one could argue that users shouldn't be shadowing those names anyways. While admittedly an edge case, this issue could increase in severity as DispatchDoctor.jl becomes more widely used or even moved to Core #40.

Possible solutions

I am personally in favor of #1 or #2, maybe #4, but not #3.

1. Compare macro equality at run-time (EDIT: or compile-time?)

I think that this can be implemented by storing both the module and name of each macro in MACRO_BEHAVIOR and returning an expression from get_macro_behavior and _stabilize_all. The expression would contain an if-statement for the union of behaviors of a given name in MACRO_BEHAVIOR and the default behavior.

EDIT: This might be able to be done at compile-time (at least, in the same way that LLVM can optimize it away), since macros have unique types.

Pros:

• Retains support for macros
• Increases extensibility
• Performance concerns are mitigated by using either @unstable or @stable default_mode = "disable"
• Fixes #42

Cons:

• Small increase in compile-time
  • Many macro expressions are small and the IncompatibleMacro case does not generate a new expression anyways
• Small increase in run-time
  • At most an if-statement with three branches

2. Treat all macros as incompatible

Pros:

• No macro is given special treatment
• No need to handle the complexity of determining macro behaviors
• Users can, in some cases, use @stable within macros
  • Macros don't compose
• Small decrease in compile-time
• Fixes #42

Cons:

• Doesn't support docstrings
• Handling macros is a useful feature
• Increased false-negatives if a user writes unstable code in the macro

3. Make macro behavior opt-in only

The MACRO_BEHAVIOR dictionary would be empty by default, requiring users to register macros as needed.

Pros:

• Same as #2 for macros that would otherwise be registered

Cons:

• Extra work for users
  • Determining the behavior for macros
    • This may be done incorrectly, resulting in more bugs
  • Repeating macro registration across projects
• If users share the same MACRO_BEHAVIOR as their dependencies (and I don't know if this is the case), then this can still result in non-extensibility

4. Make macro behavior toggle-able or customizable with preferences

• Can be combined with previous options, and would share their respective pros and cons.

[jakobjpeters]

I just realized #2 wouldn't support docstrings, oops!

@stable begin
""" foo() """
foo() = 1
end

[MilesCranmer]

For 1, would this mean having an if statement over all macros — at every occurrence of a macro — which only gets evaluated at runtime?

[MilesCranmer]

Maybe another option is to allow a per-module register_macro!? This is done in DynamicQuantities to allow per-module unit symbol mapping

[jakobjpeters]

For 1, would this mean having an if statement over all macros — at every occurrence of a macro — which only gets evaluated at runtime?

Unfortunately it might, but I'm not sure and am hoping to get it to compile away. I'm still working on a prototype, which is admittedly difficult. Returning an expression like I mentioned doesn't work. My next idea is to use downward callback functions to simulate modifying the expression without actually touching it so that the branch requiring the run-time macro can do it's thing without interfering. I believe this approach is similar to how Haskell handles impure functions by pretending to do work but evaluating it later.

Maybe another option is to allow a per-module register_macro!? This is done in DynamicQuantities to allow per-module unit symbol mapping

So the preferences file would additionally store the module and the macro would pass in __module__? Seems like a good idea!

[jakobjpeters]

Assuming the modules can be recognized and we store the macro in MACRO_BEHAVIORS with getproperty(parent_module, name), we might be able to dispatch on the macro with get_behavior(macro) itself, which is unique per module, and then I think the compiler will just toss out the unneeded methods?

EDIT: The methods would be generated from the table, whereas the dispatch might be able to occur at compile-time since macros can be returned as values from the expressions

julia> macro f() Symbol("@doc") end;

julia> typeof(@f)
Core.var"#@doc"

[MilesCranmer]

By the way, I’m not sure Preferences.jl makes sense here, because Preferences is more for compiler flag-like stuff. But declaring incompatible macros seems like it should be a constant and thus live in the code.

[MilesCranmer]

@ven-k how is @register_unit in DynamicQuantities able to only affect the local module's unit registry? I never really understood why that works. I'd like to do something similar here with a @register_macro.

[jakobjpeters]

Am I doing this incorrectly? Maybe it could be done by having the macro check whether the table already exists in the current module and define it if not?

julia> module A
       using DynamicQuantities
       @register_unit MyVolt 1u"V"
       end;

julia> module B
       using DynamicQuantities
       @register_unit MyVolt 2u"V"
       end;
ERROR: LoadError: Unit `MyVolt` is already defined as `1.0 m² kg s⁻³ A⁻¹`

I'm working on #1; still trying to fix tests but it might be viable. EDIT: The DontPropagateMacro method is incorrect here, but it shows the concept.

julia> Base.remove_linenums!(@macroexpand @stable @show 1)
┌ Warning: `@stable` found no compatible functions to stabilize
│   source_info = :(#= REPL[30]:1 =#)
│   calling_module = Main
└ @ DispatchDoctor._Stabilization ~/Documents/programming/forks/DispatchDoctor.jl/src/stabilization.jl:40
quote
    var"##435"(::DispatchDoctor._Interactions.IncompatibleMacro) = begin
            begin
                Base.println("1 = ", Base.repr(begin
                            local var"#196#value" = 1
                        end))
                var"#196#value"
            end
        end
    var"##435"(::DispatchDoctor._Interactions.CompatibleMacro) = begin
            begin
                Base.println("1 = ", Base.repr(begin
                            local var"#197#value" = 1
                        end))
                var"#197#value"
            end
        end
    var"##435"(::DispatchDoctor._Interactions.DontPropagateMacro) = begin
            1
        end
    var"##435"(DispatchDoctor._Interactions.CompatibleMacro())
end

[jakobjpeters]

Option #1 only works for the return value of expressions, which is a pretty big limitation. But it was fun to try it out at least