Lite Nyxt (or "Towards a resilient web?")

[Ambrevar]

There are two related topics I'd like to discuss here.

First, Nyxt is not lightweight, nor is it ever going to be as long as it depends on WebKitGTK (or Blink). To give you numbers, guix size webkitgtk returns something in the ballpark of 1.4 GiB (that's the closure size, so webkitgtk + all its dependencies). Now you could argue that the Guix packages are not optimal so we could shave off some MiBs here and there, but I doubt we could bring it under 1 GiB. And guix size qtwebengine is about 1.5 GiB.

There is also the issue of performance and hardware requirements. Web renderers are notoriously performance-hungry, and having every single element of our interface depend on JS is overkill. To give you a concrete example: try opening a prompt buffer while the main buffer is displaying a duckduckgo search (with JS enabled), performance is abysmal. Of course this is probably a bug that WebKitGTK could fix, but in the end these renderers are not meant to display multiple windows side by side, and this is going to be an issue...

What can we do about it?

1. We could write an alternative web engine, something fully in Common Lisp. There is already https://github.com/dym/closure, we could start from there. With the recent advances in performances that SBCL got, it could be doable. But that's a shitton of work and we are fighting an uphill battle because websites are built with Chrome compatibility in mind and nothing else.

2. Or... We could take the opposite angle and try to go sober and forget about JS support. This opens a whole new world of possibilities.

   • With no JS requirement, no need for WebKitGTK, and we can even ditch GTK. We could depend on Cairo, Pango, SDL and friends, that's about it. For audio / video support maybe depending on mpv is lighter than GStreamer.
   • We would still need to render the CSS though. Any idea if a library already exists for this? Otherwise implementing it ourselves is still a much easier job than implement a full-blown web renderer.
   • What about UI? Maybe we could still leverage HTML interactions via the https://github.com/40ants/reblocks-ui or https://github.com/rabbibotton/clog libraries (I have never tried them). Or we could go with https://mcclim.common-lisp.dev/. In any case, it would be much lighter and more Lispy than leveraging JS.

The lightweight and sober angle goes hand in hand with our recent support for Gopher / Gemini and the "small web".

Which brings me to the second, related topic: If we manage to pull off the above, maybe we could advertise Nyxt as a lightweight browser for a more resilient web (and computing in general, if we manage to reach the dream of https://github.com/atlas-engineer/nyxt/issues/887). Then Nyxt could provide features such as measuring the load of the visited web pages, reporting heavy pages, offer services like local web hosting and file sharing, bring RSS / Atom back to life, proper resource filtering, local proxy cache. For communication, we are already working on XMPP / ActivityPub, etc. which could also prove much lighter than the popular web-based social networks. The list is endless...

I'm a bit rambling here, but I thought I'd share this before I forget about it.

Thoughts?

[CorruptedVor]

I like the idea of not supporting JS. This improves

• privacy by avoiding the fingerprinting methods that rely on js
• performance by avoiding js
• security by avoiding js
• size by avoiding js

For libraries, well you could use one

• of the multiple small, simple libraries, in example http://lexbor.com/roadmap/
• that is already used in a somewhat large browser, in example what netsurf is using http://git.netsurf-browser.org/netsurf.git

edit: https://github.com/google/gumbo-parser looks good in terms of completeness

[aartaka]

Yes, NetSurf and w3m (terminal-based Nyxt, can you imagine? Ｏ(≧▽≦)Ｏ Should be quite a challenge porting all of our nasty prompt-modifying JS there, though...) can be alternative engines there!

On the note of Closure resurrection... Maybe it's not that bad implementing our own engine if we have enough people/courage? Kraken seems to thrive, even though its Git history is only three years old (¬‿¬)

[Ambrevar]

Also found http://www.litehtml.com/ with its https://github.com/litehtml/litebrowser-linux. The engine is 10k lines of C++.

[grpineiro]

I have a question here. What would be the downsides of dropping javascript support? Counting that (fortunately or not) more and more people are making websites in the form of a SPA.

If you need to use javascript, could nyxt support it in the form of a plugin/extension?

[Ambrevar]

Nyxt is built with multi-renderer support in mind. So it could be possible to have a buffer running in a "lite" engine for JS-less pages, with other buffers running WebKitGTK for JS-enabled pages.

[Anomalocaridid]

That would be a pretty neat approach. Maybe we could even go a step further and split Nyxt into a "base" browser module and an optional web engine module.

That way, if you don't want the bloat of WebKitGTK and don't need JavaScript support, you could just install the base module and not the web engine module. But if you do need JavaScript, the web engine module would still be an option.

[Ambrevar]

Absolutely, and we already have the build-time modules for free thanks to ASDF.

[Ambrevar]

A possibly lighter alternative to McClim: https://github.com/shirakumo/alloy.

[aadcg]

One of the things that Nyxt is missing to be a platform is the ability to run in non-graphical sessions. In that sense, it would be great not to necessarily rely on something as heavy as WebKitGTK.

With respect to closure, it's too soon to go down that road. It could be used as a source of inspiration to write a CSS engine though.

If we ditch GTK, I'd expect that we will have a hard time making our keybinding logic invariant. You know, for instance, that Emacs on a tty and on GTK has some disparities with respect to keybindings.

While Nyxt anticipates what might come next and the design is modular (renderer agnostic, etc), I think we should invest most of our efforts on having a single robust "instance". Later, we (and our users!) can help us port it to other infrastructures.

[Ambrevar]

One of the things that Nyxt is missing to be a platform is the ability to run in non-graphical sessions.

You mean something like lynx? Otherwise we have a headless mode and a REPL ;)

[Ambrevar]

If we ditch GTK, I'd expect that we will have a hard time making our keybinding logic invariant. You know, for instance, that Emacs on a tty and on GTK has some disparities with respect to keybindings.

Ditching GTK does no necessarily mean ditching X or Wayland :) I am personally against TTY support, I do not see the point in it. As long as we can drive it over the network (e.g. via SSH), I believe we're good!

With X / Wayland, something like https://github.com/lispgames/cl-sdl2 would be enough to handle the keys in a way consistent with GTK / Qt / you-name-it.

While Nyxt anticipates what might come next and the design is modular (renderer agnostic, etc), I think we should invest most of our efforts on having a single robust "instance". Later, we (and our users!) can help us port it to other infrastructures.

Totally agree here, I just meant to start a discussion, collect ideas and pointers. We cannot afford any of this right now :)

[aadcg]

One of the things that Nyxt is missing to be a platform is the ability to run in non-graphical sessions.

You mean something like lynx?

Yes.

Ditching GTK does no necessarily mean ditching X or Wayland :)

Correct. But McClim doesn't depend on X, or does it?

I am personally against TTY support, I do not see the point in it. As long as we can drive it over the network (e.g. via SSH), I believe we're good!

Fair enough!

With X / Wayland, something like https://github.com/lispgames/cl-sdl2 would be enough to handle the keys in a way consistent with GTK / Qt / you-name-it.

Cool!

[Ambrevar]

But McClim doesn't depend on X, or does it?

It does, at least on GNU/Linux.

[hendursaga]

One of the things that Nyxt is missing to be a platform is the ability to run in non-graphical sessions.

See https://github.com/atlas-engineer/nyxt/issues/1196 for a related issue, although I suppose technically speaking the framebuffer isn't necessarily non-graphical..

[hendursaga]

An incredibly bold idea would be integrating something like the audio web browser Rhapsode so we neither need a GUI nor even a TTY I suppose. It's been a year or so since I've tested Rhapsode myself (it's written in Haskell, gasp!), but I believe it incorporates ideas like aural icons ala Emacspeak etc. At any rate, they purposely do NOT implement JavaScript..

[hendursaga]

What would be the downsides of dropping javascript support?

Another thing to consider would be that hints etc are currently implemented using ParenScript, so we'd have to support an alternative way to do that with whatever GUI (or audio!) library we end up with.

[hendursaga]

Oh, and it appears LiteHtml uses the Gumbo parser library noted above.

[Ambrevar]

Another thing to consider would be that hints etc are currently implemented using ParenScript, so we'd have to support an alternative way to do that with whatever GUI (or audio!) library we end up with.

Many things are currently implemented using Parenscript, but few really require it. In particular, our nyxt/dom parser improves by the day, so it might be doable already!

[hendursaga]

Many things are currently implemented using Parenscript, but few really require it.

Really? I haven't caught up with all the code changes yet!

Edbrowse is another project to draw inspiration from, I suppose. I like the idea of being able to manually (as well as automagically) updating the "rendered" text when JavaScript is enabled.

[aadcg]

It does, at least on GNU/Linux.

Oh, take back my concerns then!

[alcinnz]

I thought I would share my advice from implementing Rhapsode:

I would advise to not support JavaScript when implementing your own browser engine. This is less about JavaScript itself (though a performant implementation is a massive ammount of effort) but more the overly object-oriented DOM. I like using a simpler HTML/XML syntaxtree which represents all HTML elements in the same structure. Besides there's unexplored space to experiment within regarding how much nicer browsers can make web surfing!

CSS I found perfectly reasonable to implement myself, and useful internally for defining how different elements should be rendered. Most engines rely on some domain-specific language for parsing the wide variety of properties, & I heavily lean on Haskell's pattern matching.

I would advise to handle webforms by extracting them from the embedding page so they can be tackled via a special codepath. I did this because it gives me more flexibility required for the mediums I like to target, but it also does wonders for code simplicity in a way that seems to be more inline with what webdevs seem to be wanting. This way all the hypertext renderer needs to handle are richtext & links, and you can more easily reuse existing UI frameworks for forms without worrying about how to shove it into your layout engine.

I'd recommend taking a look at Weasyprint, it has some neat tricks you might want to emulate.

A JavaScript-free engine might not work on the dominant websites, but in my experience that's relatively few sites & there's vastly more sites which it would work on. Providing an option to switch to WebKit or another dominant engine can address this, as can integrating https://farside.link/ out of the box.

I hope this helps, I'm keen to see what you come up with!

[Ambrevar]

Thanks for your insights, @alcinnz, this is very helpful!

[gravitational-dark-light]

@Ambrevar Have you head about QuickJS by chance? It is an embeddable js engine that could be used in closure without making it big or complex, and that's while being ES2020 compatible

I also would like to mention The Sciter Engine, which markets itself as a HTML/CSS/JS renderer that does UI for native apps, and happens to use QuickJS as the UI scripting language, and it's also embeddable

Thing is, I'm yet to find how make sciter run on guix, whereas quickjs is already packaged

[Ambrevar]

Thanks a lot, I didn't know about them and they look very promising! QuickJS is indeed very light, some 7 MiB apparently.

About Sciter, I could try to package it, but I have no experience with pawn (I suppose it's hosted at https://github.com/pawn-lang).

[Ambrevar]

Other sources:

• Netsurf is famous for its own embedded web engine. Maybe something we can reuse?
• The Gemini browser Lagrange uses its own SDL2-based toolkit. Looks very sleek.