Switch to backend-generated UI

[umputun]

Just an idea I'd like to explore and will apricate feedbacks from both backend and frontend devs.

Currently, we have a decent separation between UI and the backend with REST-like API. This is all fine and standard; however, in multiple places the separation bites us, and we are forced to do odd things, like infamous sed preparing HTML. Another issue is two separate places to configure things. And the last problem - we have a constant backlog of UI tasks, and the situation doesn't get any better. One more thing - with an endless number of various npm modules involved in the build/deploy/ui-functionality we are constantly racing against security issues and have to update things way too often just to address new vulnerabilities.

From another hand, our UI is not that advanced — all we need to do just 3 "widgets" with a minimal level of interactivity. Counter widget is trivial, last comments kind of static thingy and the primary comments tree already delegates some complicated parts (like preview generation) to server. The ideal solution could be fully static HTML generated by the server. This is very tempting and, theoretically, can be extremely fast, CDN friendly, and all other goodies we get from the static content. The issue with this solution - it may require a significant amount of extra generation, i.e. we will need all HTML for all sorts, all pages, and different pages/views for the different access levels. It could also be challenging from auth/security POV.

I think entirely static rendering will be hard to do right and may limit us significantly. It is why something in between, namely htmx feels promising. We could render UI components dynamically on the server-side with this library without the need for the second language (js/ts) and a dedicated UI framework. Well, htmx is a kind of js framework but sort of а different beast. The idea of such a "dynamic server-side rendering" is something unusual these days, but not necessarily a bad thing. I'm not sure how all of this can be integrated, and if we could use it as-is or write some extensions, but generally speaking to me, this direction looks very promising.

cc: @akellbl4 @paskal

[umputun]

I have very limited practical experience with htmx. All I did was a small project with a relatively simple, but dynamic UI for a monitoring/orchestration system. My impression after this project was mostly positive. It can be used and allowed me to make the frontend (bootstrap based) without much headache.

Some problems I run to:

• Initially, it feels very unusual to write component controllers returning HTML snippets and rendering from templates
• The right structure still somewhat unclear, I mean how to organize things, where to put controllers, templates, and so on
• Sometimes it is not easy to pick the right scope for the component
• Stateful parts of UI need to keep the state on the server-side somehow

[paskal]

As we won't get rid of our API anyway, I think it's worth trying to implement that on a separate endpoint as an experiment.

I have a lot of concerns which can't be answered before we have a working prototype, I don't trust pure model as htmx describe it on the main page: I saw that they themselves have an example with input verification which is done on a client: Using htmx plus plain JS for clearly client stuff could be the sane alternative to the complexity of the UI we have at the moment.

[akellbl4]

I always support keeping things as simple as possible but our comments logic really complex and huge (rendering comments tree, updating and managing comments, etc). Widgets with last comments and counter are small and simple.

What we'll be diffrent:

• We will lose testing environment
• immature technology
• htmlx size is 9kb against preact 3kb, but maybe our final code would be smaller
• htmlx uses custom HTML attributes (minor thing but I don't like such an approach) What we'll keep:
• almost the same number of production dependencies (we could reduce them already, just need to pay some time for it)
• we will keep some build stage for frontend as well just because we will need to bundle our code from modules to a single file (or creating chunks) and transpile it and obfuscate it for the smallest size

Other thoughts about htmlx: I haven't seen any project on it, their examples really simple as well (they could be implemented with pure js in 20 lines)

Arguments above just the first cons that came to me from the first look.

"dynamic server-side rendering" is something unusual these days

I agree server-side rendering is a great thing and almost all of the projects what I know use JS for it. I use SSR every day on my work project. It works on NextJs and renders my React (in our case Preact) application on server.

PS. Another huge con is the support of internationalization.

[umputun]

Good to hear the other side ;) A few questions:

We will lose testing environment

What does it mean? Testing of what exactly? The code for the actual components will be the regular go code which we know how to test

almost the same number of production dependencies

Not sure I get it. What dependencies did you mean? Let's say we can achieve the goal and have no-js (or almost no js) code involved and everything dynamic (as well as static) will be handled by the server. Unless I'm missing something big here, I don't see why we will need any dependencies

we will keep some build stage for frontend as well just because we will need to bundle our code from modules to a single file (or creating chunks) and transpile it and obfuscate it for the smallest size

I don't understand this one either. What exactly will we need to build? If all components made as controllers + templates the only build-related part (for binary deployment) will be packing templates with statik or smth similar.

Just to clarify: I'm not considering htmx approach as some kind of node.js replacement rendering UI, but rather as a helper to provide the dynamic functionality we have/want. This is very close to the good-old JSP but with a little bit of ajax (not even sure if this term in use anymore). Maybe this goal is too ambitious, not sure. But what makes me optimistic - even now server-side doing a lot of things related to the presentation, for example, it renders the tree of comments. It is not exactly in the form we can use in the browser but rendering to divs not such a big difference from the hierarchal json we currently serve.

To me, the most challenging part is how to deal with state management. I think such a thin client may have some issues to keep the state and communicate it back, but probably this can be resolved.

[umputun]

htmlx uses custom HTML attributes (minor thing but I don't like such an approach)

I've read somewhere about the use of data (or something similar) attributes instead of the custom set. Not sure if this is related to what you hade in mind, but in that discussion peps considered it as a more standard way

[umputun]

Another huge con is the support of internationalization.

what is the issue with internationalization? I can think of multiple ways to achieve it on the server-side, quick googling gave me this https://phrase.com/blog/posts/internationalization-i18n-go/ for example.

[akellbl4]

What does it mean? Testing of what exactly? The code for the actual components will be the regular go code which we know how to test

Now we have tests for frontend components and we test their behavior, some sort of unit tests for UI. But I think we could switch to E2E tests they are more complicated to write but more reliable.

What dependencies did you mean?

Okay, I haven't understood htmlx fully from the first look. It's a bit different than I thought. Looks like we will need only this lib and all our logic will migrate to HTML with attributes what could be quite difficult to support. I think we could make a small example to find out and one of the complex things in my opinion it is the login form:

• Dynamically hiding and showing icons
• Dropdowns
• Steps (Credentials step and then token step)

I've read somewhere about the use of data (or something similar) attributes instead of the custom set. Not sure if this is related to what you hade in mind, but in that discussion, peps considered it as a more standard way

Yep, data-* is a good way to put some sort of custom attributes to DOM.

what is the issue with internationalization?

If all interface always renders on the server it isn't the issue.

The summary of this comment: Rendering everything on the server it's could be good and bad at the same time. First of all, because we need to request every part of the interface from the server or we need to write some JS to show/hide or change something in the interface on the frontend (again maybe I'm wrong and because I read docs of htmlx shallowly). The messiest thing that I found is validation (js right in HTML attributes) https://htmx.org/docs/#validation At the same time, SSE looks interesting but I very curious how it could work for a huge amount of comments. Last but not least I can see how their approach smashes the wall because they have an extension for rendering templates from JSON server responses (https://htmx.org/extensions/client-side-templates/) maybe it was created for some sort of compatibility and I'm wrong. htmlx also works differently from modern SSR approaches because now it works like render the first request as HTML with JS templates and then request JSON and render it with the same code on the client-side. In the end, I want to say If we use htmlx it will be "no js" just from a code point of view :) because no-js means no js at all, but we will use JS lib which operates by HTML attributes. And yep I could support this approach just because I'm interested in writing some Go code instead of JS/TS :)

P.S. I don't have strong points for denying this approach but also I don't have them to support it. I could see htmlx is an interesting but a really controversial approatch.