Apply `esc` to returned node of `@json` to allow referring local variables

[thautwarm]

Also, you should use eval of the macro caller's module, which could be accessed with __module__ in macro definition.

Evaluation in a loaded module should avoided for it's purely dynamic.

[davidavdav]

I have been highly confused by eval which, according to the docs, evaluates in global scope, but then, in a module, that is the global scope. Or something or other. But thanks, it seems that Core.eval(__module__, x) works in some cases, but in other cases __module__ is not defined. I am afraid I simply cannot get the hang of macros---I am convinced this is what I need, basically for obtaining the AST of the julia statement, but after that I am lost... I am not sure what happens when you call functions from a macro body, and when stuff gets evaluated. Some thing like

module mod
macro ee(x)
  x
end
end

seems to

• evaluate (I didn't call an eval) when called from the REPL
• do this in the context of mod

[thautwarm]

I do understand you, for it might be quite annoying at the very beginning(even if you have exprience with macro processing in other languages). Just remember these tips:

• Macros are functions that map AST into another AST.

• Evaluating a regular julia object inserted into ASTs just returns itself.

• Each module has an eval. If you refer to a non-qualified eval, it's the eval from current object, e.g., in REPL, eval is Main.eval.

• Current module could be accessed via @__MODULE__.

• __module__ and __source__ can only be accessed inside the body of a macro's definition.

• a macrocall returns an AST that'll be then inserted into where it's(the macrocall) invoked, but if the returned AST of the macro is not wrapped with an esc, it'll be treated as the hygienic macro and prevent you from accessing the scoping where the macrocall is invoked.

function() __module__
end # wrong
macro f()
   mod = __module__
   :(1 + $(mod.a))
end
a = 1
@f 
# => 2

macro g1()
   :(1 + x)
end

function f(x)
     @g1
end
f(2) # `x` is not defined!

macro g2()
   :(1 + x) |> esc
end

function f(x)
     @g2
end

f(2) # => 3

[davidavdav]

Thanks, I will study this a few times over.

The funny thing is that by just half understanding, I can already accomplish quite a lot myself, but I keep getting surprises. Right now I can process the dubious @js { a.b: 3 } (which is not even allowed in javascript, but I believe Mongo allows similar constructs), but @js { a: b.c } gives me a somewhat un-evaluated

Dict{String,Any} with 1 entry:
  "a" => :((Dict{String,Any}("c"=>3))["c"])

and the same problem crashes @js { a: b.c + 1 }...

[davidavdav]

Hello again @thautwarm,

I've read the metaprogramming docs about 5 times now, and despite the complicated examples and your explanations I still can't make a trivial macro that returns the input, untouched, from within a module.

I understand that I need to modify an AST expression, and return an Expr, but if I can't even not modify the incoming expression I don't see how I can even begin thinking about modifying the AST. The best I can come up with for an identity macro is

module Identity

macro identity(e)
    ret = id(__module__, e)
    dump(ret)
    ret
end

id(m, e::Expr) = e
id(m, s::Symbol) = :($m.$s)
id(m, x) = x

end

but this stops working when e::Expr has symbols that refer to variables in the calling function.

So I don't really understand why my code mostly works, I call functions from within the macro that happily create dicts and stuff.

Generally, I don't understand why the macro expansion per se wants to implicity evaluate the (manipulated) expression in its own module environment just before returning the expression to the calling code.

[davidavdav]

Okay, I've found more docs and I think I've found a more concise solution to the identity macro:

module Identity

macro identity(e)
    return :($(esc(e)))
end

end

Perhaps this macro still evaluates the expression e---although I don't know if you can tell the difference between

• the macro evaluating the expression (there is a $(), after all)
• the macro returning the original expression untouched, and the calling context evaluation the expression

Perhaps there is a macro for that. Macroexpand(Main, quote @identity x end) suggests that indeed an unevaluated expression is returned.

[davidavdav]

I think I have a reasonably working version committed with now. See runtest.jl

[thautwarm]

Hi, David, it seems that you've undertood them all. The expression generated inside an esc is nothing more than hand-writting the return expression literally.

However, :($(esc(e))) equals to esc(e)... You're an interesting man haha.

module Identity

macro identity(e)
    return esc(e)
end

end

[davidavdav]

Wow, that it could be that simple! Perhaps this could be mentioned somewhere near the docs of esc or in the explanations of macros... Apart from that silliness, I've improved the package quite a bit, and it takes all kinds of obscure javascript shortcuts like @js { a, b, c } = dict for doing assignment-by-key-correspondence.

[thautwarm]

Okay, I think this package could be pretty useful in the future, but currently Julia is not that popular, which I think to be the reason why you didn't earn a lot of stars. IMO, you could do more about JSON, like (de)serialization, GraphQL, etc. You can now step further, and many(like me) would be willing to join the development of this package for it's sufficiently simple, useful and elegant. However, you shouldn't make a new feature without taking it into synthetic consideration.

[davidavdav]

I think there are already great JSON packages, like LazyJSON (haven't tried it, but needed it often in the past). This package was partly inspired by an old discussion and my recent job activities involving typescript and mongo, where everything really seems to be about almost ad-hoc constructing and deeply digging into complex JSON-like objects.

I didn't know graphQL, but I see that there is a Julia package as well. Do you mean that the route AST -> graphQL query could also be covered by macros, such that you could write:

@schema
type Query {
    hello: String
 }

and produce a Julia-representation of the query in DIana?