Pattern-generating macro cannot use `when`?

[Qqwy]

Environment

• Elixir & Erlang/OTP versions (elixir --version): Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] Elixir 1.8.1 (compiled with Erlang/OTP 20)

• Operating system: Linux Manjaro

Situation

I have written about this problem on the forum before I started to expect it was a bug here.

Take the following macros. They are meant to slightly reduce boilerplate in common pattern-matches (like in with-ladders).

defmodule Example do
    defguard is_ok(x) when x == :ok or (is_tuple(x) and elem(x, 0) == :ok)

  defmacro ok() do
    quote do
      x when is_ok(x)
    end
  end
end

ok() could then be used like so:

iex> require Example
iex> import Example
iex> # so far, so good...
iex> res = case {:ok, "Test"} do
           ok() -> "Yay!"
          _ -> "Failure"

Expected behavior

When running this example, I'd expect res to equal "Yay!".

Actual behaviour

** (CompileError) nofile:1: undefined function when/2
    (example) expanding macro: Example.ok/0

Even stranger, when trying to use it inside another module:

defmodule Bar do
  require Example
  import Example
  def foo(x) do
    case x do
      ok() -> "Yay"
      _ -> "Fail"
    end
  end
end

compilation of the project fails with this error:

== Compilation error in file lib/bar.ex ==
** (CompileError) lib/bar.ex:6: cannot find or invoke local when/2 inside match. Only macros can be invoked in a match and they must be defined before their invocation. Called as: x when is_ok(x)
    expanding macro: Example.ok/0
    lib/bar.ex:6: Bar.foo/1
    (elixir) lib/kernel/parallel_compiler.ex:208: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6

---

It seems like when a macro is passed to a case statement (or other pattern-match location; ok() = {:ok, 1}, match?(ok(), {:ok, 1}) also fail with the same error) that the macro is not properly expanded.

Oddly enough, when running a = quote do ok() = {:ok, 1} end; Macro.expand(a, __ENV__) it does not seem like the macro is expanded at that time, but only when the match is invoked later.

[NobbZ]

          _ => "Failure"

Can we assume this fat arrow beeing a copy and paste issue rather than a syntax error? I didn't realize it when posting my reply on the forum…

[josevalim]

Patterns (matches) do not allow guards. You can't write x when is_list(x) = .... The same way you can't write def foo(x when is_list(x)). Therefore one it won't work if you "hide" the when inside a macro There was a discussion to allow so in the mailing list some time ago but we did no go forward with it due to the implications in terms of matching.

[Qqwy]

@josevalim However, guards are allowed in case-clauses and with-clauses. You can manually write x when is_list(x) and it will compile and work as expected.

[josevalim]

Yes, but case clauses are defined as match [with guards]. Still, in a case clause, you can't do this:

{:value, your_ok_macro()}

It just doesn't compose the way you are thinking it does.

[Qqwy]

I do not expect that it would compose, but I did presume that a macro to the LHS of a -> would be expanded to a match [with guards] rather than just a match.

Doesn't this line in elixir-clauses.erl mean that the LHS of a -> is not expanded at all? Or does that happen elsewhere?

And even if this is expected behaviour, the compiler error message is quite confusing.

[josevalim]

It definitely expands on the left side of ->. I agree the error message could be amended though to say it only allow macros and certain expressions.

[Qqwy]

I am confused by the following:

iex> require Example
iex> import Example
iex(3)> q = quote do 
...(3)> case {:ok, 1} do
...(3)> ok() -> "Yay!"
...(3)> _ -> "Fail!"
...(3)> end
...(3)> end
{:case, [],
 [
   {:ok, 1},
   [
     do: [
       {:->, [], [[{:ok, [context: Elixir, import: Example], []}], "Yay!"]},
       {:->, [], [[{:_, [], Elixir}], "Fail!"]}
     ]
   ]
 ]}
iex(4)> q2 = Macro.expand(q, __ENV__)
{:case, [],
 [
   {:ok, 1},
   [
     do: [
       {:->, [], [[{:ok, [context: Elixir, import: Example], []}], "Yay!"]},
       {:->, [], [[{:_, [], Elixir}], "Fail!"]}
     ]
   ]
 ]}
iex> q == q2
true #huh?

I would expect Macro.expand here to expand the ok() on the left hand side of the -> (in other words, expand the [{:ok, [context: Elixir, import: Example], []}]), but it seems like it is kept as-is, and only expanded when the code is actually evaluated (e.g. Code.eval_quoted(q2)).

What is going on here?

[josevalim]

A macro inside a quote is not expanded when the quote is built. Also. Macro.expand does not traverse the tree. It expands only the root node.

[Qqwy]

Very interesting.

Expanding the quote ourselves as follows gives the result I would expect:

iex> q3 = Macro.prewalk(q, fn elem -> Macro.expand(elem, __ENV__) end)
iex> Code.eval_quoted(q3)
{"Yay!", []}

The one thing that confuses me, however, is that this is not the same as the result that is obtained when Elixir expands the macros automatically when the quote q is evaluated.

[josevalim]

Right, it is going to work in case as it does allow when at the root level side by side, but it is not a general construct. --

José Valimwww.plataformatec.com.br http://www.plataformatec.com.br/Founder and Director of R&D