Trying to append a Tree to a Frame from a Plug - Output data no longer available, please reevaluate this cell

[mcintyre94]

For context, I'm trying to use Livebook to build a UI for testing webhooks. The basic idea is that I create a Frame, and then start a Plug server that appends its requests to the frame. A minimal Livebook showing the issue is here: https://github.com/mcintyre94/livebook-kino-tree-repro/blob/main/livebook-tree-repro.livemd . Note that since it starts a port on localhost:4000 it'll need to be run locally, so I haven't put up a hosted version.

The frame is managed by a module:

defmodule FrameManager do
  @frame Kino.Frame.new()

  def frame do
    @frame
  end

  def append_to_frame(data) do
    Kino.Frame.append(@frame, Kino.Tree.new(data))
  end
end

And the Plug uses this append_to_frame to append the JSON body of the request it receives:

defmodule MyPlug do
  import Plug.Conn

  def init(_opts) do
    # initialize options
  end

  def call(conn, _opts) do
    with {:ok, body, _} <- Plug.Conn.read_body(conn),
         {:ok, json_body} <- Jason.decode(body) 
    do
      FrameManager.append_to_frame(json_body)
      conn
      |> put_resp_content_type("text/plain")
      |> send_resp(200, "Hello world")
    else
      _ -> conn
      |> put_resp_content_type("text/plain")
      |> send_resp(400, "Invalid request")
    end
  end
end

But when I send a request to this (eg Req.post!("http://localhost:4000", json: %{hello: "world"})), instead of appending the data to the frame I get this message: "Output data no longer available, please reevaluate this cell". This displays after ~2s.

Obviously there are a few moving pieces here, so I've tried to narrow down the issue a bit. If I remove the Kino.Tree.new then the data appends as just dictionary objects correctly:

This makes me think that the issue is not in Plug, though I may be wrong and apologies if I am!

If I remove the Plug, and just call FrameManager.append_to_frame directly, then I can add the tree:

This makes me think the issue is because of crossing process boundaries perhaps?

But if I create a GenServer and have that append to the frame, it works correctly.

defmodule AnotherProcess do
  use GenServer

  @impl true
  def init(_opts) do
    {:ok, nil}
  end

  @impl true
  def handle_cast(:anything, _state) do
    FrameManager.append_to_frame(%{hello: "from genserver!"})
    {:noreply, nil}
  end
end

{:ok, pid} = GenServer.start(AnotherProcess, [])
GenServer.cast(pid, :anything)

At this point I'm a bit stumped! I'm not sure why I get this "Output is not available" message but only when it's a callback from Plug. I think it's a bug in Kino.Tree.new so I've opened the issue here, but I definitely might be wrong!

[jonatanklosko]

@mcintyre94 the reason is that kinos are generally tied to the process (or cell evaluation) that creates them (to avoid memory leaks). So in this case, when you create Kino.Tree in the Plug process, it's data is removed as soon as the Plug process finishes.

As a workaround, you could have a separate GenServer (could be the frame manager), the Plug sends the info to that GenServer, and the GenServer creates Kino.Tree and appends to the frame. Since the GenServer is continuously running, the Kino.Tree data is kept.

[mcintyre94]

Thanks @jonatanklosko! I was just about to edit to say I'd spotted the workaround of using that GenServer I was testing with in the Plug handle_call :)