unable for macro to identify if argument was interpolated

[clarkevans]

In a future (breaking?) change of Julia, could you consider making it possible for a macro to identify if a substring was interpolated or not.

In some contexts, users have an expectation that interpolated substrings are treated differently from non-interpolated substrings. For example, in Mike Bostock's Hypertext Literal format, interpolated values are escaped while those that are not interpolated are passed along untouched. Julia's macro system comes close to letting us emulate this functionality, as we can tell that an argument has two parts. However, we have no way to tell which part was interpolated or not, A$("B") and $("A")B are reported identically in the abstract syntax tree.

julia> macro m(e) dump(e) end

julia> @m("A$("B")")
Expr
  head: Symbol string
  args: Array{Any}((2,))
    1: String "A"
    2: String "B"

For a suggestion of an differentiation, consider @m("$("A")") which provides an Expr and not a String like @m("A"). Then also consider the AST for @m("A$(f())").

julia> @m("A")
String "A"

julia> @m("$("A")")
Expr
  head: Symbol string
  args: Array{Any}((1,))
    1: String "A"

julia> @m("A$(f())")
Expr
  head: Symbol string
  args: Array{Any}((2,))
    1: String "A"
    2: Expr
      head: Symbol call
      args: Array{Any}((1,))
        1: Symbol f

Hence, a suggested AST with differentation could be...

# This is currently not what is produced.

julia> @m("A$("B")")
Expr
  head: Symbol string
  args: Array{Any}((2,))
    1: String "A"
    2: Expr
      head: Symbol string
      args: Array{Any}((1,))
        1: String "B"

Interestingly enough, if this AST is created manually, its quoted form is exactly "A$("B")". So, perhaps this isn't even a breaking change?

julia> Expr(:string, "A", Expr(:string, "B"))
:("A$("B")")

This change could be used to increase usability among our users. Concretely, for our use case, it is inconsistent escape the value returned from a variable, foo = "A&B", e.g. @htl("$foo") if we do not also escape @htl("$("A&B")"). That said, in this use case, @htl("A&B") should not be escaped. Since we can't, in general, differentiate these two cases, we either can't use function style macros, or we must have inconsistent behavior in our user facing interface. Thank you for your consideration.

julia> versioninfo()
Julia Version 1.5.1
Commit 697e782ab8 (2020-08-25 20:08 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Celeron(R) CPU J3455 @ 1.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, goldmont)

[JeffBezanson]

Interestingly enough, if this AST is created manually, its quoted form is exactly "A$("B")". So, perhaps this isn't even a breaking change?

Oh, this is quite clever! My initial reaction is that this would be fine.

[clarkevans]

This ticket is blocking (minor, but visible) functionality in HypertextLiteral.jl (written in collaboration with Pluto.jl folk). To explain the situation, I've written a ticket for how this deficiency manifests: https://github.com/clarkevans/HypertextLiteral.jl/issues/1

[JeffBezanson]

From triage: we will add the extra Expr(:string, ...) around interpolated literal strings. Other interpolated expressions don't need to be changed, so this will be fairly safe.