Proposal: Yew LiveView

[onelson]

First, a brief disclaimer.

I have never used Phoenix's LiveView but it's a thing that exists and seemingly solves a handful of problems seen with the status quo of modern web development. If you have experience working with LiveView or, even better, are familiar with the internals, I'd urge you to guide the discussion here. I'm still catching up when it comes to the nuts and bolts.

---

What is LiveView?

As best as I understand it today, the idea is clients receive a complete (server-side) rendered page with their first request.

Included in this HTML document is a script that manages communication with the server: as DOM events that would modify the DOM are fired, the script pushes messages to the server, where updates are processed. The server then sends new versions of the DOM back to the client, and the client uses something like morphdom to "patch itself" and therefore paint the new document state.

To me, a striking aspect of the LiveView project is there is barely any competition - why should Phoenix users get all the fun? Perhaps through Yew, we can deliver a similar workflow made stronger by Rust's wonderful type system. I'd hope by having a 2nd implementation of this idea we could breed some healthy competition and/or innovation.

What would need to be built for a Yew LiveView?

Server-side Rendering

Based on the description above, this proposal certainly dovetails with #41 since SSR would have a huge part to play.

Client State Management

According to the LiveView marketing material, their system scales horizontally and vertically which tells me they are tracking client state on the server, but I don't yet know how. At any rate, this is an aspect to research. Since Yew isn't a web server, I suppose one approach might be to develop a set of traits that different server crates could use to participate in the client/server DOM patching process.

Template Helpers

In order to get clients to request DOM updates via the server, we'd want some convenient way to write client-side code that can participate in the client/server DOM patching setup. I don't know what this would look like, practically. More research required.

---

Edit: I should have pointed out that this list of items "to be built" is likely incomplete. This is off the top of my head.

[zoechi]

To me this looks like a workaround for Elixir not running in the browser (not compile to JS), so it's unclear what the benefit would be for Yew/Rust.

[onelson]

I'm not sure that's the whole story.

While reading https://macwright.org/2020/05/10/spa-fatigue.html I started thinking about this, though I got there in a bit of a roundabout way. I was already thinking about ways to pair React redux with websockets, and serde enums for message passing between client and server. Ideas presented in LiveView took this further by moving almost everything to the server, including the rendering.

I think part of the motivation is along the lines of:

• DOM updates can be rendered faster with server-side assistance.
• Need for "code splitting" negated by an easier page-based app organization.
• less developer context switching (the entire app appears to be "backend" code).
• tighter enforcement of client/server contracts (though LiveView et al aren't required for this - just that client and server share common types for request/response bodies).

Even if Phoenix could run in the browser for the purpose of a SPA on the client, a server would still be needed for many tasks for persistence, data validation, etc.

While I don't particularly subscribe to this viewpoint, I think some folks enjoy "not needing to write any javascript" when they turn to Phoenix LiveView. I'd argue The audience for Yew would overlap with this demographic a healthy amount.

[teymour-aldridge]

This issue should be labelled "proposal"

[teymour-aldridge]

TL;DR – Sounds like an interesting idea, but I think it should go in a different crate.

I agree with @zoechi on this.

As best as I understand it today, the idea is clients receive a complete (server-side) rendered page with their first request.

This is a good idea! It's currently in the works.

According to the LiveView marketing material, their system scales horizontally and vertically which tells me they are tracking client state on the server, but I don't yet know how. At any rate, this is an aspect to research. Since Yew isn't a web server, I suppose one approach might be to develop a set of traits that different server crates could use to participate in the client/server DOM patching process.

I don't think any system implementing this should be built directly into Yew. Extension crates, etc, are all good ideas but I think that they should be added onto Yew's core, rather than baked in directly. This means that a variety of approaches, stacks and protocols can all be used. If we picked one, I fear it might be to the detriment of all the others.

participate in the client/server DOM patching process.

I'm not exactly clear on what this means, but I don't think that the server (beyond server-side rendering) needs to get any further involved in the DOM. The client can send and receive data from the server, but the browser should be sorting out the DOM – not the server (in my opinion).

[onelson]

I don't think any system implementing this should be built directly into Yew. Extension crates, etc, are all good ideas

Totally. Precluding traits that may need to be implemented alongside types owned by Yew, I could see this work happening outside of the existing crate structure.

I'm not exactly clear on what this means,

The way LiveView appears to work is to wire event listeners in the DOM that would normally introduce state changes so they instead pass messages back to the server over a WebSocket**, then state updates are reconciled and rendered server-side before being passed back to the client so it can "patch" the DOM with the update. When I wrote "participate in", this is what I was referring to.

but I don't think that the server (beyond server-side rendering) needs to get any further involved in the DOM. The client can send and receive data from the server, but the browser should be sorting out the DOM – not the server (in my opinion).

Your opinion is not uncommon! This is the status quo when it comes to SPAs today. Still, I think Yew is poised in an interesting position and could be used something akin to LiveView as well.

I think in a way you could think of LiveView as incremental server-side rendering. Instead of SSR for the only the first paint, you call back to the server for incremental updates.

Dan Abramov mentioned on twitter (in reference to https://macwright.org/2020/05/10/spa-fatigue.html) how having the client take on routing and templating were big regrets for the React team. In this, part of what's interesting about LiveView is to move more responsibility back to the server (rendering and routing included).

Something to keep in mind for this discussion is LiveView is mostly a parallel use case to SPAs. I don't think it's a replacement for them. I can think of a data-intensive application where it could be advantageous to work with the data on the server and only push display information to the client as server-state changes. Sort of like X11 forwarding over ssh, maybe.

** Using a WebSocket seems to be what LiveView uses, but I don't see why we couldn't support plain HTTP (at the cost of some additional latency). This is the oddest part of the arrangement to me, but maybe it's fine.

[teymour-aldridge]

The way LiveView appears to work is to wire event listeners in the DOM that would normally introduce state changes so they instead pass messages back to the server over a WebSocket**, then state updates are reconciled and rendered server-side before being passed back to the client so it can "patch" the DOM with the update. When I wrote "participate in", this is what I was referring to.

I think the client should work out on its own what it should do. Having to make a round trip every time a button is pressed seems to be a bit excessive (particularly for people with unreliable/poor quality internet access).

Something to keep in mind for this discussion is LiveView is mostly a parallel use case to SPAs. I don't think it's a replacement for them. I can think of a data-intensive application where it could be advantageous to work with the data on the server and only push display information to the client as server-state changes. Sort of like X11 forwarding over ssh, maybe.

I personally think that if the client needs to be in sync with the server's state it should send a message to the server, and display a "waiting" message before it gets a response. This provides a smoother experience than having the server directly manipulate the DOM.

Having the server manipulate/store a copy of the DOM is further restrictive because the DOM is a tree, and a subset of available algorithms work on tree data structures. This separation between client and server makes things easier because the client and server can choose how to communicate in a way which suits them. Server middlewares, database drivers, etc, etc are not designed with DOM trees in mind.

Still, I think it's an interesting idea which has potential but works for a subset of applications which fit this sort of model.

[onelson]

Having to make a round trip every time a button is pressed seems to be a bit excessive (particularly for people with unreliable/poor quality internet access).

I also share this concern, but apparently Phoenix have been at least somewhat successful in delivering a good experience with this model. I'm not sure they have anything intrinsic we are missing.

Having the server manipulate/store a copy of the DOM is further restrictive because the DOM is a tree, and a subset of available algorithms work on tree data structures.

I fear this is making some assumptions about the Phoenix LiveView implementation that I'm not sure are true. The impression I got was client state is modified via message passing over the socket, then run through the template code to produce rendered output which is reconciled by morphdom on the client.

They mention partitioning the document into live (dynamic) and static portions so as to give morphdom a smaller haystack to work through.

I don't they maintain a copy of the DOM on the server and manipulate it over time. I think it's closer to maintaining a "store" in redux terminology (I'm not sure what the equivalent concept is for Yew), and re-applying it to the template as it changes.

If the application of template data to the template is efficient for the initial render (SSR) then why not for incremental updates after the fact?

I can tell you have understandable doubts, so I won't continue to beat the dead horse. I just thought this was an interesting idea to explore.

[teymour-aldridge]

I can tell you have understandable doubts, so I won't continue to beat the dead horse. I just thought this was an interesting idea to explore.

It's definitely an interesting idea – I'd be interested to see if it works.

If the application of template data to the template is efficient for the initial render (SSR) then why not for incremental updates after the fact?

It's efficient because the server can render the HTML content. The size of the HTML content is very small (compared to the whole app), so it's fast to load that. The client, however, takes a long time to download the application's code. Without SSR the client would spend that time looking at a blank screen. Once the client has the code however, it should be just as efficient to render things on the client as on the server. Even if it is faster to render on the server, there's still additional overhead because rendered content has to be transfered to the client.

Anyway, this is just my take (I may be wrong) – if you think it's a good idea then my suggestion is that you try and build an "MVP" (minimum viable product) to showcase how it would work.

[JosephLenton]

In the past it was common to have a thin client which would call the server to render some HTML, and then replace some of the page with that HTML. I know them as rendering 'fragments' (as you are asking the server for a fragment of a page).

A major downside is that interacting with the UI now has the overhead of a network request. It is noticeable to users. The server also cannot do everything. You will also still need client side components. For example a carousel will need logic to animate moving from one pane to another.

There are tricks you can use to get around that. It's never as good as being able to run components client side.

It would also be a bit strange if Yew shipped as both a client side rendering framework, and a server only rendering framework. I think it would have a bit of an identity crisis.

[onelson]

In the past it was common to have a thin client which would call the server to render some HTML, and then replace some of the page with that HTML.

GitHub itself worked this way for a long time, though I'm not sure about today; "pjax" patterns. There was a recent article on "turbolinks" from Basecamp where they are doing essentially this for their primary navigation - page transitions are intercepted and xhr fetches pull down pre-rendered html from the server, then finally patched into the dom. I'm not sure the past is as distant as all that.

There's been a lot of consideration for the single user experience in this discussion, and that's more than fair for a framework specifically targeting the SPA space. I just hope we are able to balance the concerns about the individual client, button presses, directly interacting with UI elements, with the scenario where the server has contributions to share, syncing state with the client that it alone doesn't manage.

Applications dealing with multiple users in a shared context, or external systems updating data over time already have that network overhead built-in so it's not really an undue burden. I still need to dive in on the Phoenix implementation to see how they balance this.

I wouldn't be surprised if there isn't some hybrid approach where client-local interactions are able to stay local, but server pushes and dom patching are still a part of it. The issue would then be the complexity of managing the two models for render/update.

[bjunc]

Hi all,

As someone with a fair amount of JS and Elixir experience, I can say that @onelson assumptions on how LiveView works are mostly accurate. I might be able to add some additional context.

High-level, state and markup are stored on the server (Phoenix). Upon page load, content can be traditionally rendered (eg. for SEO). A socket connection is made, where UI events (eg. clicks) are streamed to the Phoenix server where they are handled. Data and DOM changes are pushed back (including non-response pushes when applicable). Sending DOM changes over the wire is inefficient, so only what is necessary to trigger MorphDOM on the client is sent. The data being sent is continually being optimized to make for leaner pushes. That said, you'd be surprised how responsive the exchange feels, even though you're making round trips (ui click, push to server, handler logic, push to client).

There are some definite pros / cons with this (IMO).

As a pro, you can write FAR less JS. It means that for many small and even medium sized apps, you could write everything in Elixir. There are "hooks" in the client lib that allow for some interoperability with JS, but I think that has been somewhat neglected and the whole process still feels pretty opinionated. I expect the client side to be come less opinionated and increasingly more extensible over time.

Downsides are that JS has very extensive tooling these days (things like page chunking, single file components, etc.). This really can't be matched by any non-JS solution. Component libraries like Material are not necessarily interoperable (it'd be difficult). I do not think there are serious efforts to build Elixir based components. However, I think there is a path forward by adding more interoperability with JS. The LiveView team maybe went a little too far with the "never write JS again" concept, and would need to pull back on that a little.

Additionally, you lose offline support, and potentially dev efficiencies with mobile apps (eg React Native, etc.).

Worth pointing out, Elixir does not really have a solid client solution. There is work on "compiling" Elixir to JS, as well as compiling a lightweight VM that can run Elixir to WebAssembly (which comes with the runtime in WebAssembly drawbacks). Since there isn't yet an Elixir solution for the client, everything really has to come from the server (to be fair, that's where Elixir really shines due to concurrency); which isn't a constraint that Rust has.

Since Rust does have a client solution, there is less of a use-case for a Rust LiveView (IMO), but I wouldn't rule it out entirely. The concept of "isomorphic" Rust (aka "universal" in JS world) could mean a combination of traditional server logic, Rust "LiveView" logic, and Rust WebAssembly logic.

Hope that helps.