Add LiveView support

[panesofglass]

Phoenix LiveView (announcement) leverages the channels mechanism to provide real-time updates to server-rendered content in web applications. In my opinion, this is one of the best parts of Phoenix.

For those unfamiliar with LiveView, here's a quick synopsis from the README:

LiveView is server centric. You no longer have to worry about managing both client and server to keep things in sync. LiveView automatically updates the client as changes happen on the server.

LiveView is first rendered statically as part of regular HTTP requests, which provides quick times for "First Meaningful Paint", in addition to helping search and indexing engines.

Then LiveView uses a persistent connection between client and server. This allows LiveView applications to react faster to user events as there is less work to be done and less data to be sent compared to stateless requests that have to authenticate, decode, load, and encode data on every request.

LiveView reduces the overhead of JavaScript client-side frameworks and libraries, aside from a DOM diff/patch tool like morphdom.

With channels seemingly well underway, might it be time to add LiveView support either in the core as a library? I started this issue following this Twitter thread. I would be happy to work on this or help out in some way.

[Krzysztof-Cieslak]

So is LiveView sending a full new version of view as an update, and then it's diffed/patched on the client?

[panesofglass]

@Krzysztof-Cieslak, from what I can tell, LiveView does a diff on the server-side by diffing the components that changed, as well as on the client-side, so it always sends the minimal amount of rendered HTML and makes the minimal amount of DOM patches, as well. It seems quite clever. My idea for a first pass was to do something a bit more like Turbolinks and have morphdom do the diff/patch on the client side alone. I don't think the Giraffe view engine is all the way there for defining components and doing diffs, but I doubt that would be difficult to add, either.

I was initially thinking of doing something a bit more like Turbolinks but exclusively with a Service Worker when I remembered LiveView. (Actually, reading through the morphdom README reminded me of LiveView).

[Banashek]

I've been curious about how this stuff works so I dug in. Here are some notes from both watching a relevant presentation as well as looking into the client-side library:

---

Presentation: https://www.youtube.com/watch?v=9eOo8hSbMAc Cliffnotes from the talk:

LiveEEx

Templates are compiled to a list of static components (string literals) and dynamic components (variables) The dynamic bits are extracted to a separate list, with a pointer in the static list The Rendered view is stored like a type RenderedView = { statics: string array; dynamics: Node option array; fingerprint: string } Cases exist for different things within the templates. These include:

• simple string values
• nulls
• complex html elements (like partials)
• iteration html elements (typical looping constructs within a template language)

Iteration elements have optimization for iteration elements (lists). It isn't explained a ton, but I'm guessing it's so that a mere append would only update a singular html element rather than re-render the whole list. I'm sure there are more optimizations you could do depending on what kind of things you send to the client (sets could use an optimization around set differences, etc).

DOM stuff

Container div (can be any element) that maintains state for both the front and back ends.

Example straight from the talk:

<div
  id="phx-xeR6nJ8t"
  data-phx-session="SQyY.g3QACZAN8XasAQ.RnkD"
  data-phx-view="MyAppWev.ClockLive">
  <!-- rendered LiveView Template -->
</div>

id attribute

• Unique ID
• Uniquely generated on every render
• Doubles as liveview channel's ID
• Whenever a liveview is connected, it will open a channel and join it

data-phx-session attribute

• Server-signed session data
• Passed to the liveview on channel join
• Includes liveview modules name, expected channel id (the same "id" as in the above id attribute), router/endpoint information (?), optional parent pid if nested in another liveview

data-phx-view

• module name of the liveview
• used as a query-selector by javascript to "find some things"
• "retrievable within a javascript hook"

Channel Details

LiveView has it's own "socket implementation", where the channels are prefixed with "lv:" So phoenix ships with a matchall that looks like channel "lv:*", Phoenix.LiveView.Channel (note that it is a unique Channel implementation)

Matching with the above example element, a singular channel might appear as lv:phx-xeR6nJ8t

When the elixir process spawns, it takes input from the Javascript LiveSocket object, including any helpful params (the example they give is the users timezone), as well as the phoenix session id (same id in the data-phx-session attribute). Mostly information so that if the process dies, it can boot back up with the same data on restart.

data-phx-session internals

• id (of the channel)
• parent_pid
• router
• session (data needed on mount)
• view (a reference to the liveview itself)

Events

Example button:

<button
  phx-click="my_click_event"
  phx-value="button's value"
>
  Click
</button>

Clicking this sends a socket message.

The message contains:

• An event (DU case, like "event" for a normal event (onclick, blur, etc), or "redirect" for a redirect event, etc)
• A Payload (which event, the type of event ("click"), the value (of the button))

Javascript details

The npm package source lives in the liveview repo in a single big ol file: https://github.com/phoenixframework/phoenix_live_view/blob/master/assets/js/phoenix_live_view.js

Looks to me like just another custom frontend morphdom-based framework that works with diffs, except it just ties heavily into phoenix. I've seen stuff like this before, but usually it just uses RPC to get diffs and shells over to some dom-diffing library. The fact that it communicates over websockets seems to be the most unique thing I see here.

Various events update data on the elements themselves: putPrivate

Before anything happens, you get some static html from the server (a placeholder until it "mounts" the data retrieved after the liveview connection is established.)

When the LiveView object is instantiated, top level event handlers are registered (things like click, keypress) (I'm guessing this is so that on these events we can check if they happen within a liveview component)

When you call .connect(), the library will:

• Find the root liveview elements with document.querySelectorAll('[data-phx-view]:not([data-phx-parent-id])') (find a liveview with no parent).
• Create View objects from these
• Join the appropriate channel for each one of these
• Re-Mount each of them with the updated data received from the server

On updates

They have the static and dynamic portions as mentioned above Only diffs for the dynamic data are sent to the client They use morphdom to update these bits

---

Final thoughts:

I like the concept of liveviews. Minimal dom-diffing over websockets seems like a neat way to add front-end functionality without resorting to a full spa framework.

My use-cases have been satisfied with giraffe + pjax-api/turbolinks + some javascript to handle websockets and the small bit of the page that requires such dynamic updates, but I could imagine scenarios in which this kinda stuff would be useful.

I'll be interested to see where their community takes this.

[panesofglass]

Thanks for looking into this, @Banashek! A lot of this still reminds me of the Blazor Server hosting model, with the biggest exception being the diff on the server-side.

@baronfel, you mentioned on Twitter that Blazor had "heaviness." Could you share what you mean by that? While I think a better tie into Giraffe View Engine would be nice, I wonder if it would be possible to set something up like this quickly with Blazor.

Another quick, initial implementation might be to avoid the LiveEE template bit at first and just send the pre-rendered HTML via web socket to the client and have morphdom make the adjustment.

Thoughts?

[baronfel]

I believe I was thinking along one of two axes:

• for client-side blazor I meant runtime heaviness. the mono wasm runtime is large. client-side blazor is not directly related to liveview in concept, though.
• for server-side blazor I meant syntactic heaviness. the razor-pages style programming model, where you've got sections of C# embedded in razor syntax, is ugly/overwrought to me.

[Krzysztof-Cieslak]

I agree that the initial stage may be done using GiraffeViewEngine (extended with event handlers), rendered on the server and pushed through channels (from server to client), with event handlers communication also going through channels (from client to server).

[Banashek]

An alternate approach that sounds close-ish to the "initial stage" you're describing: https://github.com/hopsoft/stimulus_reflex

Essentially turbolinks, except websockets + stimulusjs.

Only adding the link to increase the sources for inspiration.

Also,

While I've done some investigation into websocket performance (and memory requirements) in dotnet core, I'm curious as to the overhead of having sockets open per client, especially when you think about how liveview will open multiple channels for each top-level live-component.

Definitely premature optimization at this point, but something to note regardless.

[Krzysztof-Cieslak]

So I think I have some understanding how to design and implement initial version... but for a moment I want to take a step back and ask the question - when you'd use it over Elmish/React/Whatever on the client? Is that just about, oh I don't want to use stupid JS frameworks or do we have any use cases where it's just better than "normal" client-side rendering.

[davidglassborow]

The server-side diffing sounds related to what @krauthaufen and @dsyme are investigating with https://github.com/fsprojects/FSharp.Data.Adaptive

[panesofglass]

Is that just about, oh I don't want to use stupid JS frameworks or do we have any use cases where it's just better than "normal" client-side rendering.

I would tend to use it when you would find a client side framework overkill but you would like to improve the response time from an otherwise server-side app.

[dsyme]

The server-side diffing sounds related to what @krauthaufen and @dsyme are investigating with https://github.com/fsprojects/FSharp.Data.Adaptive

This approach allows end-to-end diffing through a programming model (it probably also allows carving off a fully static part too by enforcing the use of applicatives for the static slice). However it is quite invasive on the programming model itself - I wrote up a prototype of what it would mean to add this to Fabulous here: https://github.com/fsprojects/Fabulous/issues/258#issuecomment-552464944. In practice recovering the diff may be simpler - I'm still undecided if it's better in the long run to limit MVU to the simple cases (recover the diff but harder to scale to massively data-rich UIs) or complicate MVU views by using things like FSharp.Data.Adaptive, but get end-to-end diffing.

[kaashyapan]

Just putting this out here for folks to evaluate. Looks very interesting.

https://github.com/servicetitan/Stl.Fusion