twilco / kosmonaut

A web browser engine for the space age :rocket:
Mozilla Public License 2.0
1.69k stars 32 forks source link

What's the purpose/motivation/future plans of this project? #6

Open swarnimarun opened 4 years ago

swarnimarun commented 4 years ago

I think I represent a whole bunch of people who might have randomly seen this project, and thought to themselves it looks cool but just where is this project headed. I find "space-age" quite vague.

But anyways I think this project deserves some explanation for goals and motivations as to whether the author plans to make it a finished browser, a toy project, something just to learn, for research, or something just for personal use(seems to be like the later).

I believe it will help prospective contributors make more informed decisions, and as such might help the project not lose sight of it's goals.

twilco commented 4 years ago

My initial goal with Kosmonaut was to learn web browser development — nothing more. However, I've had a lot of fun working on it, and the project has made a small amount of progress, so I would love to find an actual useful niche to work towards fitting into. A "finished" engine à la Chromium, Gecko, WebKit, etc. is impossible, but a greatly limited feature subset or niche could be in reach.

One idea is to only implement a subset of HTML and CSS, ignoring older techniques like floats which make fast layout really hard. IIRC, the Servo developers have encountered quite a bit of pain with floats in their parallel layout implementation.

I also think qutebrowser (and others) made a really interesting niche for itself as a keyboard-driven browser. This is more of a browser chrome feature while Kosmonaut is a rendering engine, but it's an example of an interesting idea in the space.

The "space-age" bit was actually a subtle self-knock on the project, as during the beginning of the project it was about as useful as browsers in the dawn of the space-age (1957, a time long before any browser existed) 🙂 . It's come a little ways since then, and this messaging will evolve as the project makes progress.

I'll make an update to the README with some of this information so others aren't confused as well. I'd love to hear ideas about niches people would be interested in seeing filled.

Thanks for bringing this up.

rikhuijzer commented 4 years ago

Have you considered the niche of browser automation? I haven’t used Selenium in a while but remember that it could be quite a struggle to link it to my browser of choice and run headless.

nafiz1001 commented 4 years ago

How about a minimal, unix-like, html processing tool?

For example: we could use curl to fetch an html file, pipe it into kosmonaut or pass the html file as argument using bash/shell, and then it will download the remaining css or javascript files to render the page.

ghost commented 4 years ago

I'd be happy to see this used as a lightweight GUI framework for desktop apps.

Kosmonaut would implement a small subset of modern web standards for layout, enabling rapid UI development for anyone who knows HTML and CSS. We can even map browser events and handlers like onclick to native code directly, with no Javascript at all.

DrSensor commented 4 years ago

Can be embedded into game engine might be a good idea. Webkit can't fit into game engine because their code base tightly coupled to it's Javascript VM. Having a way to layout and stylize HUD and game menu with HTML and CSS might attract people in gamedev to collaborate.

NetOpWibby commented 4 years ago

A Servo-based alternative to Electron is something I've wanted for years but the Servo team has zero interest in pursuing such a thing (I asked). Glad I stumbled upon Kosmonaut, it's the closest thing I've seen that could fill the role.

amandasystems commented 4 years ago

Another interesting use case would be as a platform for experimentation. For example, would it be possible to make something like the web but using something other than JavaScript, for example Datalog or some other non-Turing complete language?

Also of course, the approach “what if we just ignore the parts of HTML that cannot be easily implemented or rendered efficiently”, as you mentioned above. It would be an interesting research question to see if you could do some sort of automated translation from “problematic” HTML or CSS into safer variants that can be efficiently rendered.

jaworek commented 4 years ago

I was recently thinking what would be possible if we could deprecate features from web browsers.

For example React Native implements something similar to display flex from CSS, but no other displays are available. This simplifies creation and management of layouts dramatically. I guess it is also much simpler in terms of implementation than what we have in browsers today.

Seems like this project can answer some of these questions. It would be very interesting to see what is possible in terms of speed, simplicity and ease of use when a lot of bloat was removed from modern browsers.

botiapa commented 4 years ago

As others mentioned, I'd love to see an electron alternative, even if it only had just a subset of features.

niutech commented 4 years ago

@evanram @manithree @botiapa We already have a bunch of lightweight alternatives for Electron: electrino, neutralino, Quark, Ultralight, Miniblink, Deskgap, WebWindow, litehtml, graffiti, yue, nodegui, etc.

As for alternative browser engines, there is Netsurf and Flow, although the latter is closed source.

What is the rationale for building yet another browser engine, instead of improving existing ones?

botiapa commented 4 years ago

I was not aware of the fact, that there were so many alternatives, to electron. My bad.

NetOpWibby commented 4 years ago

@niutech All those projects you liked use the same browser engines we're used to. This one uses Servo. I think that's enough reason for yet another one.

niutech commented 4 years ago

@NetOperatorWibby No - Flow, Netsurf, Quark, Sciter, Ultralight, Nodegui, Litehtml, Graffiti - all use their custom rendering engines.

NetOpWibby commented 4 years ago

@niutech Regardless, I think having a Servo-based one is great for the ecosystem.

niutech commented 4 years ago

@NetOperatorWibby Is it really based on Servo? From the readme:

Kosmonaut's current implementation is heavily inspired by Servo, sometimes taking code directly from it.

This is not original Servo engine, but yet another spin-off, increasing fragmentation. Why not use the original Servo, which loads full web pages via HTTP(S), unlike Kosmonaut?

NetOpWibby commented 4 years ago

@niutech If it did, would your opinion change?

niutech commented 4 years ago

@NetOperatorWibby This discussion doesn't go anywhere. My point is, Kosmonaut is yet another web engine, inspired by Servo, which only increases fragmentation, without any specific use case. Instead of reinventing the wheel, it would be better develop Servo itself, which is way more mature, advanced and open source too.

One specific use case for Kosmonaut could be a lightweight web engine for embedded systems, but even here we have existing solutions: WPE, Ultralight and Netsurf.

Treevs commented 4 years ago

@niutech I think it's worth pointing out that Kosmonaut is a project that was started to learn about building a web engine from the ground up, rather than a project started to replace anything that already exists. As of right now, this is a for fun/learning project that happened to pick up steam on HN and Reddit (and it wasn't @twilco that shared it on either platform).

NetOpWibby commented 4 years ago

@niutech That's why I directed your line of questioning back to the point, which helped. As I stated in my initial post, the Servo team has zero interest in creating an Electron-like solution. Kosmonaut being at least inspired by Servo will be the closest thing we've got. This project doesn't need a "specific use case" either, it can exist for more than one thing.

I think your argument of "we've already got these other projects so this shouldn't exist" is in poor taste. Fragmentation exists everywhere so I find that point invalid as well.

niutech commented 4 years ago

@NetOperatorWibby Actually, there is a POC of Servo-based Electron alternative called ion. Shouldn't it be easier to reuse existing Servo engine than to build a new one?

Edit: there was also Graphene and Servoshell, both inactive.

NetOpWibby commented 4 years ago

@niutech It'd be better to have something fully formed, not a proof-of-concept last updated three years ago. Ion also requires knowledge of Rust programming. Electron does not.

pie6k commented 4 years ago

Would be great to see browser engine that drops support for huge amount of legacy, but is as fast as mobile apps.

It amazes me how much faster apps are, when rendering something that looks and behaves the same way.

shmalebx9 commented 4 years ago

simplest use case to go after is those looking for a lighter webengine. Bloated browsers are always a good backup but something that can read the news without a lot of overhead could be useful. Browsers using the engine could also implement an older user-agent to get a more simplified version of the page for Kosmonaut to parse. For example: I use an old nokia user-agent to get a version of CNN which loads about 8 times faster than their standard view.

Aeres-u99 commented 4 years ago

In these times I see electron almost everywhere, imagine being able to become a lightweight replacement for electron and then being able to render things in small ways? One more thing thats on top of my head these days is a lightweight browser for the preview of Markdown, the current solutions are pretty heavy (yeah I wrote a documentation but should I be running a full fledged web browser to see basic links, images? I think not) kosmonaut will be ideal in such a case. If html parsing could be done effectively, we can even create a html parser using kosmonaut which would allow us to edit and write html syntax selectors in realtime.

niutech commented 4 years ago

@shmalebx9 For a lightweight web browser, why don't you use K-Meleon (based on Gioanna), Otter Browser (based on QtWebKit), Qutebrowser (based on QtWebKit/QtWebEngine), NetSurf (own engine), Ultralight (own engine) or even Opera Mobile Emulator (based on Presto, download here)? You could spoof the user agent string in every web browser out there. Why reinvent the wheel instead of improving existing solutions?

Aeres-u99 commented 4 years ago

@niutech there is lot of issues with these browsers, K-meleon is windows only, Otter and Qute arent really light on 15+ tabs, Netsurf is amazing in fb mode, but normally its ui is not great. The problem with these browsers is the way they were created, I feel that in the way of implementing features they got too bloated, for instance take surf for example. Even though the source code is <1K sloc, but its still not good or light. Rendering is a mess and no new or variation exists in terms if engine. All other browsers are just adaptation of Mozilla or chromium, meaning you either love chromium or Mozilla. Webkitgtk has been found insecure there by making surf, vimb, lariza,luakit and many others insecure by design. Reinvention is necessary because thats the only way we can avoid repeating the mistake our predecessors did and make a truly lightweight Browser. Which is light not just in code, performance but also in operation and features.

bbqsrc commented 4 years ago

It could be useful as a modern replacement for Dillo, a rather old FLTK browser that implements a very light subset of the web.

MRobertEvers commented 4 years ago

It seems to me that the browser space is pretty crowded; whether it be Electron alternatives, or full grown browsers, they are everywhere. I think this is the unpopular opinion here, but I agree with @niutech. A full fledged browser, even if 'lightweight', really looks like an insurmountable task to make it secure and useable for even a fraction of websites.

Since this is a learning project, perhaps experiment a little? Just throwing out a couple ideas below: (I'm aware these things have been done before.)

If the goal is main-stream adoption, I don't think it will be successful.

NetOpWibby commented 4 years ago

I really don't understand this mindset.

The same argument ("crowded space", "already exists", &c) could be made for almost everything. That doesn't mean something isn't worth working on. This feels like the projection of one's inability to pursue such a project.

MRobertEvers commented 4 years ago

"It's not worth working on" isn't the argument. At least that's not my angle. There are existing avenues to work on browser development and it's perfectly fine to want to keep learning and developing a web browsers.

Since this project is looking for direction, this is an opportunity to provide a venue for something other than (stereotypical) browser development - which seem to be a dime a dozen.

immackay commented 4 years ago

@evanram This is a great idea! Would you be willing to scope out some of the necessary architecture changes and open an issue outlining a potential roadmap?

ghost commented 4 years ago

@evanram This is a great idea! Would you be willing to scope out some of the necessary architecture changes and open an issue outlining a potential roadmap?

No

CharmCityCrab commented 4 years ago

I saw the invitation on the front page to offer achievable niches for this project here in this issue.

Here are some ideas:

  1. Most browsers dropped support for the gophernet protocol (gopher://) a long time ago. I can't really argue with them removing it, since it's barely used and seemingly outdated technology that goes back 30 years.

However, Wikipedia shows the number of gophernet servers as having nearly tripled over the last few years. I really have no idea what people are doing on gophernet these days, but clearly some people are doing something, and maybe this project could create a more modern secure client to do whatever they're doing.

Since it's old technology, it wouldn't be a hard protocol to support, it might be fun to code, and you could look at whether there are any innovative ways to display the protocol better than old web rendering engines used to, without making any changes to the actual protocol or standards.

I'm kind of approaching this from the perspective that your goal here is to mess around with things, have some fun, and gain experience.

I can't defend what I'm talking about as being useful to many people, but there are probably at least a handful of people in the world who would be happy to see some better gophernet support if you or someone else developed it as part of your rendering engine and put some sort of basic web browser around it. Of course, those clients would also include the rest of your rendering engine with it's limited http:// and https:// support, so it'd be a way of getting the rest of your work out there more, too.

  1. Another thought is, could you merge what you've done with an existing browser engine? The idea would be that anything that you've done would render pages when it can, but it would automatically fall back to Gecko, Chromium, or whatever when it can't, thereby giving people a workable web rendering engine to base browsers on that is neither purely Gecko or purely Chromium, and that would become less and less Gecko or Chromium as you build your thing out more and more.

I don't know if this is technically feasible or not, I'm just throwing it out there.

  1. If you're just looking for something to hack on, and don't mind it being totally unrelated to this project, there is someone trying to do a fork of the new Fenix version of Firefox for Android who is basically going at it alone and could use some help here:

https://github.com/interfect/fenix

I just thought I'd throw that in there in case you're in search of something to do as much as a use for this project. I don't see how this project could actually be folded into that project.

shmalebx9 commented 4 years ago

@shmalebx9 For a lightweight web browser, why don't you use K-Meleon (based on Gioanna), Otter Browser (based on QtWebKit), Qutebrowser (based on QtWebKit/QtWebEngine), NetSurf (own engine), Ultralight (own engine) or even Opera Mobile Emulator (based on Presto, download here)? You could spoof the user agent string in every web browser out there. Why reinvent the wheel instead of improving existing solutions?

I've been using qutebrowser as my mainstay for about a year now. Your point is entirely valid, and qutebrowser is the closest browser I can find to the use-case I just mentioned (besides possibly surf). The issue is that most browsers, qutebrowser included, eventually scale up to be able to handle all the fanciness of newer web standards. a webengine with a lighter footprint could allow for a simple browser for things like reading the news/IOT/embedded etc. As far as I can tell, Qtwebengine supports as much as chromium or firefox.

niutech commented 4 years ago

@CharmCityCrab There are plenty of existing Gopher clients. There is even a Gopher successor called Gemini with a lot of clients.

@immackay For an existing lightweight cross-platform app engine using HTML, have a loot at: graffiti, litehtml, Azul or Sciter (this one is closed source, but freemium and has Rust bindings). What are you lacking in these?

cztomsik commented 4 years ago

Web browser complexity & overhead mostly comes from:

You probably want such features in a browser and Gecko/WebKit are excellent implementations (and they should be rather improved instead of starting new projects). If you want to do a browser I think you will inevitably end up with something as complex as what there is already and so I think it will be pointless.

I think it's best to keep doing whatever you want as long as it's fun but don't waste time trying to do impossible.

And for electron, it's complicated. Embedding (real) web browser, including all of that is bad, that is true and you could replace it with libhtml/netsurf/rmlui/xxx but you'd miss some features because it wouldn't be webkit anymore (and it could be very limiting).

So it all depends what is the "subset" you're aiming for and I think that is really hard question - it took me 2 years to get some idea of what I want to do with graffiti (my experiment, still not ready).

zacharytyhacz commented 4 years ago

I'd be happy to see this used as a lightweight GUI framework for desktop apps.

Kosmonaut would implement a small subset of modern web standards for layout, enabling rapid UI development for anyone who knows HTML and CSS. We can even map browser events and handlers like onclick to native code directly, with no Javascript at all.

This is most upvoted here, and I agree.

Even for linux with xorg having something like this to create small GUIs with html/javascript-LIKE native code to handle clicks, styling, that sort of thing and be something that I - myseld would love to use.

niutech commented 4 years ago

Even for linux with xorg having something like this to create small GUIs with html/javascript-LIKE native code to handle clicks, styling, that sort of thing and be something that I - myseld would love to use.

Isn't it what QML, ClutterScript or Proton Native are already providing?

cztomsik commented 4 years ago

Kosmonaut would implement a small subset of modern web standards for layout, enabling rapid UI development for anyone who knows HTML and CSS. We can even map browser events and handlers like onclick to native code directly, with no Javascript at all.

Note this is exactly what graffiti is about. Major use-case is currently deno (then backport to nodejs) but it will be available also as rust lib too (I don't think anybody would use it alone but it could be very interesting with something like html! macro).

ceuk commented 4 years ago

I'm really late to the party but I'd love to see a browser that is purely focused on rendering HTML email. Given the basic nature of email markup vs the rest of the web, it would enable one to focus on a small subset of features (tables, images, basic typography etc).

As to the "why", I can see two benefits:

1) It could potentially be much faster than a conventional web-browser for viewing HTML emails and similar content ( useful for things like email/patch-based workflows and anyone that reads email in a terminal)

2) It could be much more opinionated than a conventional web-browser (and ideally much more extensible/scriptable)

It's incredibly niche, but the proposition was for something niche, and the increasing prevalence of HTML email represents a significant frustration for certain use-cases and workflows.