Thoughts on the underlying OS

[kraileth]

The main point of this whole project seems to be offering a nice way for just about everybody to use a computer. I do not have any need for a polished desktop personally but I applaud visionary thinking and courage to try reaching for the stars! For that reason Hello is pretty appealing to me nevertheless.

What I'd like to point to are a few of FreeBSD's shortcomings. Don't get me wrong, this is not another "you should just use Linux instead!" topic. On the contrary; I'm using FreeBSD as the sole OS on my computers, I truly love it and I think that you made a good choice. But there are things that require work. Things that do not have any direct impact visible on shiny screenshots but can mean a lot to how the system feels as a whole. And no, I'm not talking about the splash screen feature being in a less than ideal state. It's things that go quite a bit deeper, more like - to the core.

FreeBSD has some weaknesses that show when it comes to the desktop. One thing is that multi-seat functionality is not available. While on Linux I can login with a different user without logging the other out, I can't do the same here. I'm a former team member of GhostBSD, had fun and learned quite a bit. But honesty demands also mentioning that the system is appealing more or less only to more technical people who know (!) what the advantages of FreeBSD are and are willing to accept the disadvantages for having them. If you are aiming for a general-purpose system for "mere mortals" there are some fields that would really require work - and ideally we should try to add more advantages over Linux to the list as well.

1. Boot time and service reliability are actually one problem that needs to be solved. FreeBSD doesn't take terribly long to boot but for people used to e.g. Linux (or Windows) it's an annoyance. A good part of that is the kernel taking much longer - and while there are FreeBSD committers who have done work in that area, there's probably a lot more to do. While on a pretty bare server system the userland part doesn't matter as much, the concept of Hello sounds like it's going to load quite a few helpers. Firing these up takes more time than it needs to due to FreeBSD's init system (rc.d) not doing parallel startups.

There has been initiatives to introduce OpenRC as an optional alternative, but it hasn't happened, yet. TrueOS (upon which GhostBSD is built these days) made the switch and as a result the desktop is ready quite a bit faster. It requires giving up things people have come to love (configuring services via rc.conf variables) but offers a couple of advantages over the old way. What it does not do however is provide reliable service management! To get that today still requires using venerable programs like daemontools. Solaris has SMF, Linux has Systemd (which merits a lot of critique but regarding service supervision it's definitely some form of progress!) - we're lacking behind here.

Recently there has been some interest in the s6 supervisor suite and the author of 66 (utils around s6 to make using it as init easier) has offered help if anybody was to seriously try getting it working properly on FreeBSD. Considering something like that would solve the problems of slow boot time and missing service reliability both for good and elegantly.

2. Not all base system components are ideal for a desktop system. Do we need things like Sendmail? BSNMP? Svnlite, portsnap and the like (if FreeBSD's packages are not to be used in the long run)? And on the other hand: Wouldn't it make sense to include zsh in base? What about making the attempt to import Xorg (or OpenBSD's Xenocara as done by the Hyperbola guys) into base? A system like Hello would never ever under no circumstances want to break the GUI! But by keeping X and the actual OS separate this is much more likely (and it has happened to me before) than if X was part of the OS.

3. Configuration is a mess on Unix-likes - and when it comes to the benefits of context-aware configuration, we're still living in the age of barbarism. I can only point to the book published by the Elektra initiative, chapter 0 alone has been a real eye-opener for me. We're wasting so much potential here by not designing our systems according to the results of that research. Hello is about things like auto-configuration? The Elektra principles could help a lot here.

These are just three points to start with. Other people do user-visible things way better than me, but if any of these (or other "under the hood" topics) would be of interest to the project I could probably contribute at least some research.

[probonopd]

Thank you @kraileth for your thoughtful remarks.

I've only been using FreeBSD since last quarter, and only because FuryBSD mainly "just worked" for me and was logical and understandable. So most likely I haven't encountered many of the less pleasant sides of FreeBSD yet, but those exist on other OSes as well. Been using Mac OS X (yes, it was still called that when it was at its peak imho) and Linux for a long time. Have tried to build a Darwin-based system with the PureDarwin project around the time when Apple switched to Intel. Still, very new to FreeBSD proper.

So your knowledge and experience of a longtime FreeBSD user and GhostBSD contributor is definitely highly welcome here!

The "Linux" (and with that, I mostly mean userland - which I put the term in parantheses) world has this mentality that everything needs to be in a distribution, and not just a distribution but many of them. The "Core OS" is not really well defined by anyone so there are many very similar stacks that still are not really compatible. FreeBSD is one system, has one ports/packages collection, and so on - this makes the test (and support) matrix much smaller. In the "Linux" world you write something that works on your system and you can be sure the other day you get 1,001 bug reports because things are subtly broken somewhere. Distributions can't agree on the most basic aspects and fight everything that was "not invented here". At least this has been my experience with developing stuff for "Linux".

There are already way too many Linux distributions for my taste. I don't think the world needs yet another one.

Boot time

Yes, Clear Linux OS makes me jealous: Cold boot to a Gnome desktop in under 5 seconds on a very modest machine. But then, it's a Gnome desktop. Which I clearly don't want. Getting hello "into" Clear Linux OS probably won't happen, and cloning the whole Clear Linux OS thing is probably way more work than I am willing to invest. So, I am OK with spending maybe 10 extra seconds of boot time for now, even though I agree that less is better. I hear that FreeBSD's init system (rc.d) will get parallel boot capabilities sooner or later...

@jordanhubbard made great points imho with NeXTBSD, but apparently the wider FreeBSD audience didn't really embrace them (as did I back in the time when he advised me that rather wasting my time with Darwin I should better hack on FreeBSD).

Service reliability

FreeBSD's init system (rc.d) seems pretty OK in this regard to me. (Will it stay this way when parallel boot capabilities are added?)

TrueOS

It's dead, right? (Unfortunately almost everything that iXsystems was involved in is either dead or changed name and direction more often than I can keep up with.)

Not all base system components are ideal for a desktop system.

True. But are those real biggies?

Do we need things like Sendmail? BSNMP? Svnlite, portsnap and the like

Probably not, but if we deactivate them, do they really bother us much? (We probably might even be able to just delete them if we are really that concerned regarding space.)

Wouldn't it make sense to include zsh in base?

Probably yes. But a preinstalled package isn't that much worse, is it?

What about making the attempt to import Xorg into base?

This. Couldn't agree more. I'd like to see this happen. Could be in an extra archive (with all its drivers) as long as it was always released alongside the base system imho. But then, as long as "no one" uses FreeBSD on the desktop, maybe it's not seen as a priority. Chicken and egg.

OpenBSD's Xenocara as done by the Hyperbola guys

Never heard of it. Need to check it out. Especially since I am sceptic about Wayland since it breaks POLA in grave ways. Does Xenocara answer the "Xorg is soo insecure" concerns of the Wayland folks?

A system like Hello would never ever under no circumstances want to break the GUI! But by keeping X and the actual OS separate this is much more likely (and it has happened to me before) than if X was part of the OS.

Very true. That being said, we can think of different models on how the hello system gets updated. I could well imagine an image-based rather than a package based approach for the hello system. Think updating iOS or updating Android. Those images could be QC'd beforehand to ensure all components inside work well together.

Configuration is a mess on Unix-likes

Yes. Apple solves this nicely with plist files. I wonder why no one else seems to embrace them. Or, say, JSON. Or, say, YAML. Well, probably because no one can enforce anything on open source developers...

It's an equal mess on Linux, though. At least ;-)

Elektra initiative

Thanks for pointing me there. Wasn't aware of Elektra.

But then, do "mere mortals" really need all that configurability? I'd make the point that excessive configurability is a) because developers are too lazy to make things "just work" without the need for configuration, and b) results in undue complexity. (I know that this is a controversial theory, but I just like thinks that work right out of the box and make a habit of judging software by its default settings.)

Again, thanks for chiming in. This project clearly can benefit a lot from users and developers like you who have a lot more experience in all things FreeBSD than myself.

[tuaris]

I just want to chime in with my thoughts. From what I've seen so far, the project has made two very good and very important design decisions.

• It's using FreeBSD. I can't express just how important and how great of a decision this is.
• Application deployment. Been looking forward to someone doing this for FreeBSD. If it's one thing Apple got right, it's self-contained apps.

That said, @probonopd please don't let anyone convince you into moving off FreeBSD. If there are problems/limitations with FreeBSD lets fix them instead of giving up. Obviously you've already taken a huge step towards doing that already with the architecture of this system.

If I had one criticism, I would say it's don't try to be too much of a MacOS UI clone. I imagine that you are building an OS that's using FreeBSD as a core, but it's not 'technically' FreeBSD. I look forward to seeing what happens. Perhaps we may actually end up with a true open source and free Unix for everyone. If you need any computing resources (for project infrastructure, updates, websites, packaging, building) or assistance with FreeBSD specifics I'm happy to offer them.

[kraileth]

I've only been using FreeBSD since last quarter, and only because FuryBSD mainly "just worked" for me and was logical and understandable. So most likely I haven't encountered many of the less pleasant sides of FreeBSD yet, but those exist on other OSes as well.

Oh yes, there are pain points with each and every system. FreeBSD is doing very well in comparison, but that doesn't mean that it doesn't have issues of its own. If you haven't been using it for too long, you've missed some of the good sides as well: E.g. updating is generally so much less of a problem due to the "whole system" concept.

At work I have responsibility for some servers that begun as FreeBSD 5.x servers and they are still doing their work happily as 12.x today. Not that long ago actually I got a customer who had a root server (because he'd rather not pay for management) and after years he needed urgent help. That beast was still running FreeBSD 7.3 (!), so I had a system technically from 2008 (which is from before I started getting into the IT professionally) and he needed a "fix" ASAP because browsers started displaying warnings due to ancient OpenSSL and all that good stuff (and of course I had to operate with minimal downtime and without breaking anything). So I updated to 8, then to 9 and converted the system over to ZFS. Then I updated to 10, started jailing the running services and transferred those jails to a fresh system with 11 (because I didn't trust an OS not updated for so long). Was a bit of effort, yes, but worked like a charm. Good luck doing the same with a, say Debian 4 machine in 2018/19!

With boot environments we have an especially great tool at hand that's only slowly coming to Linux land (due to the Trident "renegades" going to Void). It can make things so much easier for the end user.

Been using Mac OS X (yes, it was still called that when it was at its peak imho) and Linux for a long time. Have tried to build a Darwin-based system with the PureDarwin project around the time when Apple switched to Intel. Still, very new to FreeBSD proper.

I've been loosely following PureDarwin since I'm interested in cross-platform packaging and never had a Mac myself. Thanks for that project! But as an outsider it's hard to judge its state. There's probably too few people involved for it to really take off.

So your knowledge and experience of a longtime FreeBSD user and GhostBSD contributor is definitely highly welcome here!

Wouldn't say that I'm a longtime FreeBSD user - about 5 years of using it seriously, a little longer if playing with it also counts. I've used Linux exclusively for about a decade before that, though.

The "Linux" (and with that, I mostly mean userland - which I put the term in parantheses) world has this mentality that everything needs to be in a distribution, and not just a distribution but many of them. The "Core OS" is not really well defined by anyone so there are many very similar stacks that still are not really compatible. FreeBSD is one system, has one ports/packages collection, and so on - this makes the test (and support) matrix much smaller. In the "Linux" world you write something that works on your system and you can be sure the other day you get 1,001 bug reports because things are subtly broken somewhere. Distributions can't agree on the most basic aspects and fight everything that was "not invented here". At least this has been my experience with developing stuff for "Linux".

Couldn't agree more. Before I left for FreeBSD I had been living in niches of the not so much mainstream distros for a while. What you describe is both a blessing and a curse IMO. A blessing because it's so easy to try out something completely new with Linux - and a curse because even if you got something really nice, there's basically no chance it'll be able to make as much impact as it should... E.g. I admire the work others have been doing with Musl, but as soon as you venture off from glibc, things start to break because just about no one cares for anything but the "standard" GNU C library. So I've kind of been fighting "Linuxisms" even before I got into BSD! :laughing: I mean, it's perfectly legitimate that developers cannot care for hundreds of distros they are not using themselves. But sometimes I have this feeling that we're falling back to the dark ages before POSIX was a thing in some regards. And that's pretty bad.

There are already way too many Linux distributions for my taste. I don't think the world needs yet another one.

Exactly! I've also been there; in 2012 I set off to start my own distro - for educational purposes, wanting to understand how that works. It was a good thing to have done some of the work, but I'd much rather help an existing project today if I wasn't completely convinced that I had a unique idea and use case (which I think applies to Hello).

Yes, Clear Linux OS makes me jealous: Cold boot to a Gnome desktop in under 5 seconds on a very modest machine. But then, it's a Gnome desktop. Which I clearly don't want. Getting hello "into" Clear Linux OS probably won't happen, and cloning the whole Clear Linux OS thing is probably way more work than I am willing to invest. So, I am OK with spending maybe 10 extra seconds of boot time for now, even though I agree that less is better. I hear that FreeBSD's init system (rc.d) will get parallel boot capabilities sooner or later...

What Intel did there really is impressive and we're a miles away from that. FreeBSD has this little problem that while it's officially a general purpose OS it really is a server OS for many of its protagonists. And since you don't boot servers quite that often... The desktop FreeBSD guys do exist and from my point of view do outstanding work. But desktop FreeBSD is a niche within a niche and it kind of shows (Again: just being honest here, no attempt to scare anybody off whatsoever!). A project like Hello might be beneficial for both sides, though. It might be worth the time to analyze the reasons for PC-BSD / TrueOS / Trident's demise and try to learn from that.

Regarding parallel rc.d: I'd take that with a grain of salt. Or rather: I'll believe it when I see it. While I haven't dug as deep into it as I'd like, from what I can say the whole concept of rc.d does not fit so well with parallelism at all. Maybe somebody comes up with very clever ideas and compromises, but I kind of doubt it. People love the current implementation for one thing: simplicity. And that's going to die with parallelism added in. Worse: If doing parallel startup we should probably rather follow a more modern approach (there are enough toys on the table!) than try to retro-fit something into old technology and risk a half-baked outcome with potentially a lot of headaches.

Maybe a middle ground is viable, though; the main problem with rc.d is probably that it's shell-based. It looks like people trying to come up with replacement technology agree that for something as important as system startup shell scripts should be avoided (projects like nosh even display it right in their name!). S6 uses execline which is a shell-like non-interactive scripting language that is more performant, has a lower memory footprint and is more secure by design. Maybe an "rc.d-elin" would be an interesting intermediary step? Just some brainstorming here, though.

@jordanhubbard made great points imho with NeXTBSD, but apparently the wider FreeBSD audience didn't really embrace them (as did I back in the time when he advised me that rather wasting my time with Darwin I should better hack on FreeBSD).

As one of the initial project founders he's a legend and I have no idea what FreeBSD (or any BSD really) would look like if he hadn't done the old pkg-* tools and the ports framework. But I think a lot of people disagree with what he did later on. I have no real opinion on NeXTBSD, didn't look into it too much. Having something like launchd might have had quite some benefits, though!

Service reliability

FreeBSD's init system (rc.d) seems pretty OK in this regard to me. (Will it stay this way when parallel boot capabilities are added?)

It isn't. Basically it suffers from the same issues that SysV init does. You start a service and then want to know if it's running, so you ask the system about the status. System replies: "Yupp, it's up and running" - but it isn't, because it has crashed. The pid file is there, though, and thus the system believes that the service is running... This is terrible, worth of one of the inner circles of Dante's Inferno! While I like projects like OpenRC for adding parallelism and such, they do not really attack this problem. You do an "rc-status" and see that some service has crashed. Then you do "service foo start" and it says that it's already started! So it's "service foo zap" and then starting again...

Solaris does much better with it's fault management. A service that should be running died? Just restart it. Died too often in X amount of time? Well, then back off and scream for the admin, there might be a configuration problem or another blocker that cannot be automatically resolved! Service supervision really is a thing if you want to have a reliable system. FreeBSD isn't doing exceptionally bad, but as a person who gets a notification and has to get up in the night (here I am at 3:00 am local... :laughing:) just because Apache had a bad day and tripped over its own legs for some strange reason (doing something special when log rotation happened or something...) I do see lots of room for improvement. :wink:

TrueOS

It's dead, right? (Unfortunately almost everything that iXsystems was involved in is either dead or changed name and direction more often than I can keep up with.)

As a graphical distro: Yes. But as far as I can tell, Eric forked it for continued use in GhostBSD.

Not all base system components are ideal for a desktop system.

True. But are those real biggies?

In terms of space required and such? Not really, they could probably be neglected. But from the perspective of having a "clean" system I think it makes sense to start throwing some stuff out.

Do we need things like Sendmail? BSNMP? Svnlite, portsnap and the like

Probably not, but if we deactivate them, do they really bother us much? (We probably might even be able to just delete them if we are really that concerned regarding space.)

I wouldn't say that it's a high priority right now (or in the near future). But eventually I'd like to have a system that's well documented and as simple as possible. And superfluous components are... well, superfluous.

Wouldn't it make sense to include zsh in base?

Probably yes. But a preinstalled package isn't that much worse, is it?

Well, a package can break with base system changes. Zsh specifically is linked against four libs in base. Let's say ncurses receives a soname bump and we restart after the system upgrade but without having updated the packages, yet. Then it's broken (can probably be "fixed" with a symlink, but still). I love zsh and use it all the time, but I'd never make it the default shell for administrative users - because it's not in base. The license is fine, though, and it's not that much of a burden. We can even configure it to behave like FreeBSD's default tcsh (e.g. what zsh calles "up-line-or-beginning-search" and what I miss so much when I have to use bash or the likes). It would be great to have it in base.

What about making the attempt to import Xorg into base?

This. Couldn't agree more. I'd like to see this happen. Could be in an extra archive (with all its drivers) as long as it was always released alongside the base system imho. But then, as long as "no one" uses FreeBSD on the desktop, maybe it's not seen as a priority. Chicken and egg.

An additional distribution set, yes. That was exactly what I had in mind. OpenBSD does this btw. (split into multiple dist sets, though).

OpenBSD's Xenocara as done by the Hyperbola guys

Never heard of it. Need to check it out. Especially since I am sceptic about Wayland since it breaks POLA in grave ways. Does Xenocara answer the "Xorg is soo insecure" concerns of the Wayland folks?

Haven't looked at it from a security perspective, but since OpenBSD ships it by default and those guys are dead serious, I'd expect that it's not worse than Xorg. :wink: Seriously though, one advantage would be that Xenocara resembles Xorg from the time before the "modular" builds. It can be built as a whole and not as dozens or hundreds of packages. How much of their hardening applies outside of OpenBSD is something that I don't know and cannot judge.

A system like Hello would never ever under no circumstances want to break the GUI! But by keeping X and the actual OS separate this is much more likely (and it has happened to me before) than if X was part of the OS.

Very true. That being said, we can think of different models on how the hello system gets updated. I could well imagine an image-based rather than a package based approach for the hello system. Think updating iOS or updating Android. Those images could be QC'd beforehand to ensure all components inside work well together.

Makes perfect sense. TrueOS had an interesting approach, too: When you started an update it created a boot environment and installed the updates to it. That way you could continue working with the old system without any problems ("Firefox needs to restart because libraries have been updated") and you'd simply boot into the new system as time permits. This also had the advantage that you could get back easily if something broke. I think that the actual OS should be read-only anyway, so an image-based approach makes sense.

Configuration is a mess on Unix-likes

Yes. Apple solves this nicely with plist files. I wonder why no one else seems to embrace them. Or, say, JSON. Or, say, YAML. Well, probably because no one can enforce anything on open source developers...

Not familiar with these. Will have to do a little research.

Elektra initiative

Thanks for pointing me there. Wasn't aware of Elektra.

But then, do "mere mortals" really need all that configurability? I'd make the point that excessive configurability is a) because developers are too lazy to make things "just work" without the need for configuration, and b) results in undue complexity. (I know that this is a controversial theory, but I just like thinks that work right out of the box and make a habit of judging software by its default settings.)

Chapter 0 of the book sums it up pretty well. In short: Context-aware configuration offers additional possibilities to the system as a whole and can make it possible to derive defaults from these context - i.e. making some configuration dispensable while still having a system that (automatically!) offers to different users what they are likely to need. E.g. based on geo-IP the system would guess that both of us would like a "de" keymap without us having to choose anything during the installer or later configuration! I don't know about you, but while it'd actually be wrong for me (I'm a de-neo² typist), it would be a good guess and actually closer than just assuming "us". And that's only the tip of the iceberg. A unified configuration resource would mean that applications have access to other application's configuration and can derive context information from that (e.g. if your desktop uses a high-contrast theme for the visually impaired other programs offering similar accessibility options might want to automatically enable them whereas on other systems it makes more sense to not enable them). Which means that there is no longer a single "default" but rather default being a context-dependent setting that is meant to be right out of the box even for the most diverse group of users. It's like admitting there is no "one size fits all" but still not giving up the idea of defaults.

I think that something like this would be an aim that fits very well with the Hello project. And the best thing is that this is not all dreaming, but the basic framework to do this is here (but little known). If you take a somewhat closer look and like what you read, it might be for mutual benefit of both projects if Hello was to do something with it, because Elektra has kind of a hard time gaining a foothold even with several programs on Linux. Now imagine what a whole system designed to take advantage of this could look like! Being FreeBSD-based, Hello also has the advantage of being able to just do something like that system-wide - which would be very, very hard if not impossible on Linux due to fragmentation and it's nature of "kernel plus a bunch of packages (by different parties!)" vs. the BSD's "whole OS" approach.

[mateuszkwiatkowski]

Hello,

I decided to add feedback to this issue instead of creating a new one. Thanks @kraileth fot this initiative, idea is great and I'll keep my eyes on it.

The Software

Nowadays operating systems are tools for running Apps and I think that the more supported apps the more users it will have. Each user have his own set of Apps that help get her/his work done. Very often these Apps are proprietary and sooner or later you'll have to support them to break through. For me the set of tools is: docker, vscode, terminal and chrome. For other guy it'll be photoshop, Intellij IDEA and firefox. I think that Proton project showed that operating system could be brought down to the level of firmware responsible for doing I/O and hardware management and it does not necessarily mean that popular Apps could not be supported. With recent works around Linuxulator running Proton on FreeBSD may not be that far away. This conclusion takes me to the second point:

The Hardware

I think that the biggest Macs' strength is PERFECT support for hardware. Every piece of engineering put into the machine is supported by macOS and just works. It's similar with Windows machines but hardware support is distributed between many hardware suppliers. When you choose to install Linux or BSD you agree to accept some inconvenience and that depends on hardware. On laptop A one can't suspend-resume, on laptop B a bluetooth device is not supported and on laptop C you can't even install your OS because of some undocumented EFI bug. There're already some efforts from FreeBSD Foundation to provide perfect support for some laptops. I don't know about their progress though.

Support

When the project will gain traction you'll receive ton of Github issues, often related to mediocre hardware. Apple have the comfort of supporting only narrow set of hardware and it is a multibillion dollar company. How could a volunteer project be able to support hundreds of computer models at similar level? I think that in order to succeed you'll need a budget and chosen set of hardware that you support. Speaking of myself, I would pay some amount of money to get well supported operating system that will help me get my job done. The Elementary project is experienced in that area and it could be valuable to talk with them and learn.

Conclusion

I have listed only a few hurdles to overcome to deliver a perfect desktop operating system. I may sound pessimistic, but I am writing all this to propose some ideas for your consideration:

• Focus on supporting Apps - this is what brings users to Operating Desktops, not special effects and candy graphics
• Choose which hardware to support - stable and supported hardware make happy users
• Talk with experienced people - there're people experienced with providing desktop experience to users. Talk to them and get feedback as early as possible. This'll help you avoid waste of resources. Make a contact with FreeBSD, GhostBSD developers and iXsystems - there're really cool people that will want to talk with you and share their experience. I'd also try to make a contact with Elementery project.

I support this project with all my heart. I'll follow it and I'll try to use the system at the nearest opportunity. Good luck!

[jordanhubbard]

Quite the long thread on this issue, I see! I'm not quite sure how to contextualize everything being discussed, particularly since FuryBSD seems to have EOL'd itself and retired and the link to daily ISO images on this project is also now dead, so I'm not really sure what the "work product" is or how to even evaluate it - the build scripts were not written to bootstrap from different platforms (like macOS, which lacks commands like realpath(1)) and the documentation on how to bootstrap all of this is not intuitively obvious, so... I guess I will simply try to make a few general points without having actually seen the tech first-hand:

1. Whatever your goals for this project, you should start by making the build machinery for it as touchless as possible if you really want to attract support. When I checked in the first version of the release targets in /usr/src/Makefile on FreeBSD, I took special pains as the release engineer to make it as easy as possible to bootstrap your own release build (and get an ISO out). Anything more complex than "make release", which automated a huge number of steps, was not going to cut it. Once I had that working, I added more "knobs" here and there, but I still kept it as simple as possible. One command and you had an ISO image to play with that represented the current source BOM (bill of materials). If I had that to do all over again, I'd have also linked the package system and the release process together really early on such that everything in binary form was a package and the ISO image was simply the sysinstall UI + a bunch of canned package manifests ("user profiles"), rather than having binary "distributions" and 3rd party packages be separate. Sure, I papered over the divide afterwards in terms of the UX, but it was all far from elegant.

2. There is absolutely nothing wrong with cloning MacOS's intention to be a "just works out of the box" solution, since that was exactly our goal at Apple back in 2001 when I joined the company. Even if you have lots of "nerd knobs" in the configuration, you still need to make it easy to install, upgrade and use or people find themselves spending too much time messing with the OS and not enough time doing whatever they actually installed the OS in order to DO. An OS should be a means to an end, not the end.

3. Don't forget to pick a unifying mission. If you try to be all things to all people from the beginning, you will almost certainly fail due to having too many objectives and not enough resources to chase them. I spent a lot of time arguing with the FreeBSD Foundation that the new mission for FreeBSD should be as a teaching OS which also comes with a perfectly serviceable desktop (not exceptional, just serviceable), a bunch of developer tools installed by default, and all of the sources you'd need to further hack on the distribution from there. My northstar in terms of user experience and HW support would be RaspberryOS (formerly Raspbian) because it has at least two awesome things going for it: 1. It runs on extremely low-priced hardware, which makes it accessible to the bulk of the world's population who cannot afford $3000 PCs but a $50 "PC" is just fine and more than good enough (and the RPI4 is a very capable little beast, as is even the PI-zero-W). 2. It includes a rich set of developer tools out of the box, e.g. python, GPIO / SPI programming libraries, etc - everything a student / hobbyest needs to be productive and engaged immediately out of the box. It even comes with a copy of minecraft and other games installed, just to make things more fun. Beautiful.

Add to that picture a fully buildable source + ports tree which lets you tinker and customize it from there, learning about the internals along the way, and you'd really have a pretty amazing OS, something Linux lacks for the most part because everything is so encapsulated and "distro-ified "that it's not even clear how to build your own images, much less random pieces of the kernel and userland from one convenient, browsable location like FreeBSD still has.

Just a bit of advice from an old guy who has been there, done that, and constantly sought to disrupt conventional thinking in whatever area he looks at, which is also why a lot of my suggestions for "NeXTBSD" were unpopular - that was very much the entire point of doing that (in point of fact the name "NeXTBSD" was a pure tongue-in-cheek joke that few seemed to actually get). If you want to really move the needle in tech, it shouldn't be about winning popularity contests so much as defying conventional wisdom and trying out new things, new approaches, and adapting to new technologies (like powerful little ARM machines that, in the case of the NVIDIA nano, can even do workstation level 3D graphics for under $100).

Good luck!

[kraileth]

I absolutely love the spirit of what @jordanhubbard describes as seeking to disrupt conventional thinking: A good part of what Unix is comes from the will to challenge the given situation (simplicity instead of complexity), the will to defy the concept of "this is how it's done" and the will to explore new ways while at the same time trying to be reasonable and have a goal of creating something useful.

Was going to write about what I see as a possible way forward for Hello, anyway. I'm still more interested in a sound underpinning for the actual desktop than the desktop UX itself. We really should think about the release process as well as the update process some more as that will lead to a bunch of interesting questions. Building FreeBSD is easy enough and installing it somewhere is also just a make -C /usr/src DESTDIR=/something installworld installkernel. Now the interesting question is: What do we do with that data? The classical way is make distribution and having an installer that will extract the selected tarballs onto the fresh system. If we're to go with a different approach (i.e. image-based OS), it might make sense to investigate how to actually get the system to disk.

Here are a couple of thoughts on that:

1. We definitely want ZFS. One way to do things could be preparing a clean OS boot environment, snapshot it and distribute that snapshot on the install media. The installer would then effectively prepare the pool on the system, copy the snapshot over and do a forced rollback.

2. If we define that the base system (minus configuration and unique data) is sacrosanct and the user should not mess with the OS image, we can provide updates in form of differential snapshots that are used to create new boot environments and switching over which is the default one. FreeBSD's updating mechanism works well on the server but is too complicated (and takes too long especially when switching to newer releases). Snapshot-based updates as proposed could solve that problem. Classical freebsd-update has to do a lot of work determining the state of the system and figuring out what to actually do - we can probably do without all of that. That would not just be a big benefit for the user but also for maintainers. HardenedBSD for example has its own updater which does not support differential updates but always updates the whole system - for the simple reason that it's much easier to maintain that instead of wasting scare developer resources on something that's effectively only "nice to have" for many people (unless you're on a slow connection...).

I like the idea of e.g. SmartOS to basically have all the drive only for the data and persistent configuration while viewing the OS as something that needs to be replaceable all the time (updating SmartOS means putting a new image on the PXE server and rebooting the running machines - a very elegant approach!). Of course in desktop space relying on PXE booting makes little sense, but perhaps this idea of strictly restricting the OS to a well-defined role ("needs to be there obviously but the non-technical user should never have to care about it at all") could still be applied. What I have in mind here is that updating the OS basically means receiving some data, generating a new BE, reboot into the new system at a point in time where a reboot is ok for the user and be done.

3. For a system updated over a long period of time we need a utility to take care of managing the available boot environments and snapshots. We don't want to keep all of them around after all, some could be converted to ZFS bookmarks, etc.

4. The idea has been that the OS should be read-only. We have to take into account that there are parts of the system that need to be read-write, e.g. /var/log. Others need to be read-write at least temporarily on first boot, e.g. for generating the SSH host IDs (for people who might want to SSH into their systems) or read-write temporarily to change configuration (if that is even desired).

5. A layout needs to be designed to make this feasible; i.e. having a self-confined operating system image and any local configuration separate from that so that an update won't overwrite your local settings. We must not forget that we should provide a utility that checks local configuration after an update and e.g. warns the user that there's a deprecated local setting that might be removed soon (and offer to correct that automatically for the less technical user).

6. I'd also like to have something like configuration profiles where you can have the system adapt to different environments to work in. E.g. "if we get an IP from this range apply that system configuration which disables wifi and enables this default printer". Things like that could help laptop users who work at different places a lot.

7. A way to keep local configuration versioned would be great so that changes can easily be reverted ("I know that it worked last week, but I don't remember what I changed since then!") would be great. Yes, I can do that with git / svn but that's neither the best way that I can think of nor is this something that should be required to do manually. The system should just do that for the user IMO.

All the things that I mentioned previously (init, what base should look like, Elektra, ...) can be tinkered with at a later time as this (system structure and distribution form) is something really basic to decide early on and I'd like to push for that first. Again, this is just a bunch of thoughts (some were on my mind for a while, though) to get a discussion going.

Also with @probonopd obviously liking what NeXTBSD tried to do, too, I propose the somewhat similar "NexusBSD" as the name for the OS component of Hello.

[jordanhubbard]

ZFS and the boot environment abstraction in FreeBSD are absolutely worth leveraging - we used that heavily in FreeNAS to support atomic updates (clone current environment, install to clone, roll-forward to clone only on complete success) and we also had our own delta-update mechanism. It wasn't accomplished through zfs send/receive since that would be too expensive (and currently not even possible) to support from a CDN, but the user doesn't ultimately care how the updates get streamed to them, just so they happen atomically and on whatever release train they're subscribed to (development, stable, experimental, etc) with the ability to hop between trains. See https://github.com/freenas/build for all the machinery to accomplish that, under BSD license. It should serve as inspiration if nothing else!

[probonopd]

Welcome @jordanhubbard and thanks for chiming in!

I'm not really sure what the "work product" is or how to even evaluate it

Here's a blog post that describes the basic ideas with screenshots.

Right now this stage is in what I would call the brainstorming phase plus a pre-alpha continuous build of an installable Live ISO.

The build system for the Live ISO is based on FuryBSD but really I see it more as "the missing FreeBSD Live ISO generation tool" than anything else: https://github.com/helloSystem/ISO/. Continuous (untested!) builds are at https://github.com/helloSystem/ISO/releases/tag/continuous-hello (but they are, to put it mildly, "still rough around the edges".

Making the build machinery for it as touchless as possible

Totally agree that it needs to be dead simple to build (and change!) the ISO. In fact, I even go one step further in that I am trying to set up things in a way that can allow everyone to just clone the project and build their own ISOs using public services (like GitHub and Cirrus CI) so that one doesn't have to think about setting up infrastructure before being able to hack on the system (even though it's perfectly possible to build on private machines of course).

One command and you had an ISO image to play

./build.sh hello ;-)

"just works out of the box" solution

Yes, you described it better than I ever could. I need to be realistic about what I can and cannot achieve with a small project like this. So 99% will be pure FreeBSD, with a very thin layer of custom UI and preconfiguration on top of it, all of which of course can be tinkered with by more experienced users.

If you try to be all things to all people from the beginning, you will almost certainly fail

Very true.

For now I'd be happy if we could make something that runs e.g., graphics applications well enough so that graphic artists can use it without getting too distracted by "the system".

that was exactly our goal at Apple back in 2001 when I joined the company

I really admire Apple's efforts to blend the best aspects of the Mac and of Unix in the early 2000s. Unfortunately I don't get the same vibes from their current products anymore, where the focus seems to be more on Memoji and locking down everything.

like powerful little ARM machines

Looking for one of those that actually run FreeBSD well including GPU acceleration yet are still affordable. @darkain has already started work on aarch64 images. But an affordable tangible device with good FreeBSD support currently seems to be the missing piece... most affordable ARM systems seem to focus exclusively on Android/Linux support. Or do you have good news for us in this regard?

disrupt conventional thinking

Been there: Got constant sh*t from the Linux community for pushing the idea of application bundles, err, AppImages (think .app inside a .dmg). Not even when Linus Torvalds said he liked the concept did it get much better...

Thanks for your advice. Your points resonate very well with me.

[davidchisnall]

This project looks great and has a lot of overlap with what we wanted to achieve with Etoile (though without the overreach that largely killed that project). A few thoughts:

Configuration

Please take a look at UCL. This is written by a FreeBSD developer and I believe it's in the base system. There was an effort in FreeBSD to port everything to being able to use UCL as a config file format. There are a few really nice things in UCL but the most useful feature from an OS packaging perspective is that it has generic merging rules that make it easy to put defaults in a read-only location and even have immutable defaults, system-wide configuration and per-user configuration, centrally managed. I think FreeBSD would accept patches to make everything in the base system use UCL so that's not a diff that you'd need to carry.

FreeBSD has an annoying requirement that /etc is on the root partition because it's where things that are needed to start init live. With ZFS, it should be possible to have an immutable /etc (or some other directory that contains the bits currently in /etc that are needed for boot).

Nexenta had a mode in apt that automatically took ZFS snapshots before any upgrade so you could roll it back. The desire to be able to support this is why the default ZFS configuration puts /usr/local in a filesystem and doesn't put /usr/local/etc in one: you can atomically snapshot everything. The fact that most ports don't do anything like UCL's immutable defaults means that you can't cleanly separate /usr/local/etc from $LOCALBASE. If you add some generic machinery to enable doing this that ports can opt into, I think there'd be some appetite for upstreaming this and for you to be able to carry patches to the ports that you most care about to support it.

Apps

As a long-time GNUstep developer, I really like the .app bundle model, but it doesn't work well for libraries. macOS and Windows avoid most of this pain by shipping a few GBs of system libraries that anything can depend on. That's at odds with providing a small base image.

The PBI model from PC-BSD was a similar idea. Effectively each app was built with a custom $LOCALBASE and all of its dependencies. The installer then used hard links to deduplicate things that were present in multiple PBIs. There was a sketch design to build something similar on top of pkg. The big down side for PBI was that installing any KDE app required you to download about 500MB of KDE dependencies. Not so bad if you just installed one KDE app, but if you installed 10 of them then downloading 5GB to end up with around 600MB of installed software was annoying.

It's impossible to separate the app install problem from the library-dependency install problem.

GUI toolkits

Cocoa is not built on top of Carbon. Both were built on top of the CoreGraphics libraries that are roughly equivalent to Cairo / Skia. They may have shared some code internally but don't assume that you will be able to layer something nicer on top if you don't like Qt in the end.

Aspects of the GUI programming APIs leak into the UI. For example, the way that OpenStep defines copy-and-paste and drag-and-drop make it easy to add things like Services, which are a huge win on macOS / GNUstep (anything you can select can be sent through filters, for example something that takes an equation in LaTeX markup and returns a PDF that then replaces the text and embeds the source in PDF metadata so the transformation is reversable).

All the cool kids are using Electron now (they probably own shares in Micron), and there is a FreeBSD port. There are also some much lighter-weight embedded browser frameworks that might be interesting and are BSD / MIT licensed. There's also Godot.

The important thing is to support things like DBUS (even if you hate it) so that things written with other toolkits.

Sandboxing

I'm not a fan of Wayland but the authors' criticisms of X11 security are valid. Any app that can connect to your X server has basically full control over any other app you're running. For this reason, if no other, I'd be inclined to consider going the Wayland route, even if you use the X11 compat mode and run every app in a separate X11 instance in Wayland.

FreeBSD has two good mechanisms that you can use for sandboxing apps. The conceptually simplest is Jails. These compose very well with ZFS: you can clone as ZFS filesystem, start a jail in that clone, let the app do whatever it wants in there, and mediate access to user files (for example by providing a FUSE filesystem that exposes the user's home directory with some access control preventing it from accessing files created by other apps unless the user explicitly authorises it). Jails have separate IP addresses, so you'd want to route them all to a dynamically configured firewall. You'll quickly see that dynamic firewall configuration is one of the big weak areas of FreeBSD.

The jail model is very nice if you can get the tooling right because it means your users can run things 'as root' if they need to, yet still in a sandbox.

The other is Capscicum, which prevents any access to the global namespace. You'd need to provide something like libpreload or Casper to explicitly grant them access to things.

XDG

You can't get away from things like this. Their rules for where files are stored are not too bad. If you want config / cache files to go away when things are uninstalled (macOS with NSUserDomain has the same problem), then you can either make things like ~/.config and ~/.cache either FUSE filesystem or something that is explicitly cleaned up.

[probonopd]

Hello @davidchisnall and thanks for sharing your thoughts.

This project looks great and has a lot of overlap with what we wanted to achieve with Etoile

Indeed there is another implementation based on GNUstep and could be made to use more components from Étoilé: https://github.com/probonopd/LIVEstep. As much as I admire the projects, sadly it seems there is not much user-facing development going on in GNUstep and Étoilé lately, possibly because there are not many develoeprs familiar with them. LIVEstep did get a few stars but no significant contributions so far. So my hope is that basing the hello system on Qt the barriers to entry will be lower for potential contributors, and more "native" applications will be available.

UCL looks interesting. Since hello will be implemented as a very thin layer of GUI and defaults on top of FreeBSD, it would have to be FreeBSD who is adopting UCL, not hello. Unlike projects like GhostBSD we would like to stay as close to the underlaying FreeBSD as possible, especially in the beginning when the project is still very small.

automatically took ZFS snapshots before any upgrade so you could roll it back

The whole upgrade story for hello is not determined yet. I could imagine a read-only Core OS filesystem image (think Android, iOS) with an overlay/union filesystem for user modifications. One advantage would be that you could have many versions of such read-only bootable images in parallel on the same partition. In case we stay with a more traditional package-based system, we'll definitely want to do automatic Boot Environments before an attempted update.

It's impossible to separate the app install problem from the library-dependency install problem.

From my experience with the AppImage project, what people want is a slowly-changing, well-tested, rock-solid Core OS with the possibility to run the latest and greatest user-facing applications. Ideally multiple versions in parallel, so that you can test things in the latest and greatest version and can always return to a "known good" one in case things don't work out properly. Traditionally, you had to update the Core OS in order to run the latest applications, for the reasons you are well aware of. I'd like to find a way to solve this. It is a tradeoff between efficiency and robustness, and with ever-increasing hardware capabilities I tend to lean more and more toward the robustness side. For Linux, there are now AppImages that bundle all dependencies (except for the kernel and Xorg), and while not the most efficient, they are not that terribly worse in terms of size than what we see on Windows and on the Mac...

The file manager in hello can handle (simplified) .AppDir and .app bundles for now, but that will not be the end of the story. More thought on this will ne needed, and I think a clever de-duplication scheme and maybe the use of fileystem images inside the bundles (similar to what distr1 and Haiku OS do, married with the drag-and-drop concept of .app bundles, could be interesting.

Wayland

Let's reconsider it once it has proper NVidia support and screen recording works out of the box without the need for GNOME or KDE. Which may well be never.

GUI toolkits

Right now it seems to me that Qt/PyQt is not the worst choice, mainly for pragmatic reasons including application availability and developer availability. But I could imagine that over time the system could adopt something newer like the Fyne toolkit, depending on how its adoption for applications goes. Or some (which?) development will lead to a newborn interest in GNUstep?

Capscicum

Could it be used to implement per-process filesystem namespaces? Then it'd be super interesting for implementing the .app bundle ideas outlined above.

XDG

Yes, we probably need to support it on some level but I consider it as a legacy liability. Conceptually it looks to me that we need something like a LaunchServices database. [More on this topic]

Interesting that you mention XDG and FUSE in the same sentence. This idea had already crossed my mind while working on https://github.com/probonopd/go-appimage for Linux, but I've never come around to implement something.

[jordanhubbard]

Well, this is probably the first time I've seen a github issue used as a mailing list. 😆

Rather than suggest a mailing list, creating a discord channel or even a free slack instance might be a more effective way of hammering out these sorts of details, but this is just a suggestion.

1. Regarding managing libraries as bundles, it's obviously totally possible to do that. I do believe that NeXTStep was the first to really make both apps and libraries first-class components with the NSBundle() API, which is to say that NeXT made the underlying directory structure merely an implementation detail. It's the API which actually gives everything from the L10N libraries to the image activator (underlying exec(2)) to the dynamic linker / compiler toolchain a way to think about bundles as an abstraction. If I were re-implementing Apple's app / framework bundle mechanism, I would definitely not focus on (or expose) the implementation, I would focus on the abstraction and then figure out how to make other tools call it.

2. All that said, once you have a stable host OS and you've figured out how to have N versions of a library and M versions of an app installed side by side without anything fighting, you remember that the rest of the world went to containers to solve this problem (also creating a security partition at the same time) for more components than just binaries and libraries but also python scripts, pip wheels, conda packages, yada yada yada and you spent a lot of time and energy just solving a problem from the 1980s using technology from the 1990s. 😬

I'm not a big fan of docker containers, but I'm old enough to realize that they won this battle. The docker image is the new app bundle.

[darkain]

I'm not a big fan of docker containers, but I'm old enough to realize that they won this battle. The docker image is the new app bundle.

This is FreeBSD based. Docker is based on containers, which is a re-imagined version of Jails from FreeBSD, but an overall worse implantation. The docker system has added some nice streamlining around it, however, containers have had some major issues too (you mention security, which they've actually been historically REALLY bad at). RedHat has moved away from Docker, and onto Podman now for example, as it doesn't require root access.

But enough Docker talk. We're FreeBSD. We have jails. The containerization of applications is already solved in the base OS. "Side by side" applications are trivial with jails, as well as isolation. Jails also offer virtualized networking, so if the UI allowed it, applications could even be behind their own NAT or bridged connection away from the host. This also means that it will be easy to add resource limitations to applications, by adding those limitations to the jail itself. Jails are also very nicely integrated into FreeBSD's ZFS file system, so snapshots, backups, replication, rollback, etc will all also be very easy to implement.

I do agree that these are solved issues, but solved in a different way.

[kraileth]

@darkain Well, I guess the point was that everybody and their dog is "talking Docker today" - and in that sense Docker has in fact won. The technical details are of less interest to the general public (or even many project managers who get to decide how money is being spent). If I didn't get it wrong, we even have TrueNAS scale today which is Debian-based simply because iX's customers kept demanding docker support. Let's face it: Even the most hard-core BSD enthusiasts are experimenting with Docker in Bhyve and stuff like that - usually because they need it at work (e.g. even Oleg Ginzburg, the guy behind cbsd and ClonOS who dares to challenge Proxmox with a FreeBSD-based system!).

There sure are ways to do with FreeBSD what most people do with Linux and I would love to see more people with enough guts and technical knowledge to attack k8s for example (Hashi Nomad jails-driver anyone?). But previous efforts to e.g. get parts of the OpenStack infrastructure working on FreeBSD failed because it was pretty much a one man show and it was given up because of a serious lack of interest from the community.

Not saying that improving that sad state would benefit FreeBSD as a whole and thus Hello as well (at least indirectly), the focus of this project is way different. I hope that after some more brainstorming the actual planning can begin and come up with a sound roadmap of things to do (and not to do).

[probonopd]

Well, an .app bundle that contains all libraries is conceptually not that far from a Docker container, especially if you run it in a Jail. Maybe we can find some clever way of de-duplicating things that are in more than one such .app bundle, e.g., by clever use of 1.) composing the .app bundle not from individual files but rather than from its packages, and 2.) OpenZFS deduplication features.

Here is a proof-of-concept:

What looks like an application is actually an .AppDir (conceptually like an .app bundle) that actually contains the following:

That AppRun file is the main entry point. It union-mounts all the packages in that directory, and finally executes the Gnumeric application.

Quick and dirty proof-of-concept code (not nice in any way):

#!/bin/sh

HERE="$(dirname "$(readlink -f "${0}")")"

PKG=gnumeric

FOUND=$(ls "${HERE}/${PKG}-"*.txz || true)

if [ -z "${FOUND}" ] ; then
  PKG_CACHEDIR="${HERE}" doas pkg fetch -d -y "${PKG}" # TODO: Download all(?) dependencies as well
fi

THERE="${XDG_RUNTIME_DIR}/RANDOM"
CHROOT="${THERE}/chroot"
mkdir -p "${CHROOT}"

DIRS=""
PKGS=$(find "${HERE}" -name "*.txz" -type f)
for p in $PKGS
do
  BASENAME=$(basename "${p}")
  mkdir -p "${THERE}/${BASENAME}"
  archivemount "${p}" "${THERE}/${BASENAME}"
  DIRS="${THERE}/${BASENAME}":"${DIRS}"
done

unionfs -o allow_root "${DIRS}"/ "${CHROOT}"

doas mount_nullfs /tmp "${CHROOT}"/tmp # for /tmp/.X11-unix

doas chroot "${CHROOT}" "${PKG}"

umount -f "${CHROOT}"/tmp
umount -f "${THERE}"/*
sleep 1

rm -r "${THERE}"

The idea is that if you have multiple such AppDirs, then OpenZFS can do its deduplication magic. (Or we might even use hardlinks).

(This works but is dog slow, because mounting many .txz with archivemount isn't exactly meant for this purpose - but imagine the same being done with tools and formats that are actually optimized for this use case, like distr1).

The downside of this approach is that packages are not very granular (do we really need a full Gnome icon set just to run Gnumeric? etc.).

macOS and Windows have a clearly defined set of desktop-oriented libraries/frameworks that you can assume to "always be there". This does not exist on Linux or BSD. And even if we had it, developers tend to always depend on newer versions than what ships with OSes. Which is why an approach like the one I am thinking about here might be useful because it allows you to run the latest bleeding edge apps alongside long-term stable ones on a long-term stable OS.

Does this make sense?

Edit Nov 2022:

#!/bin/sh

HERE="$(dirname "$(readlink -f "${0}")")"

PKG=gnumeric

FOUND=$(ls "${HERE}/All/${PKG}-"*.pkg || true)

if [ -z "${FOUND}" ] ; then
  sudo pkg fetch --output "${HERE}" -d -y "${PKG}"
fi

THERE="${XDG_RUNTIME_DIR}/RANDOM"
CHROOT="${THERE}/chroot"
mkdir -p "${CHROOT}"

DIRS=""
PKGS=$(find "${HERE}/All/" -name "*.pkg" -type f)
for p in $PKGS
do
  BASENAME=$(basename "${p}")
  mkdir -p "${THERE}/${BASENAME}"
  fuse-archive "${p}" "${THERE}/${BASENAME}"
  DIRS="${THERE}/${BASENAME}":"${DIRS}"
done

unionfs -o allow_root "${DIRS}"/ "${CHROOT}"

sudo mount_nullfs /tmp "${CHROOT}"/tmp # for /tmp/.X11-unix

sudo chroot "${CHROOT}" "${PKG}"

umount -f "${CHROOT}"/tmp
umount -f "${THERE}"/*
sleep 5
mount
rm -r "${THERE}"

Works but is dog slow due to fuse-archive. But imagine this with more suitable, image-based and kernel-mounted filesystem images instead of tar.xz packages...

[grahamperrin]

FreeBSD PkgBase

From https://github.com/helloSystem/hello/issues/56#issuecomment-751333939 about shells:

… since helloSystem is for now primarily a live ISO that's of no concern — to the user. …

From https://github.com/helloSystem/hello/issues/1#issuecomment-624223293 about the system as a single file:

… a system should be "human manageable" … The user should be in control to decide whether to update, when to update, how many old versions to keep around, etc. …

With an assumption that helloSystem will continue to support pkg-based installations and updates: I do think that PkgBase will – eventually – be a good fit. One way or another it will make management of system updates more user-friendly.

Additional context: #27

[grahamperrin]

https://github.com/helloSystem/hello/issues/37#issue-744062806

… a few of FreeBSD's shortcomings. …

From my point of view, in the context of helloSystem being simple … and usable:

• sound
• Bluetooth
• Wi-Fi
• last but not least, sleep and wake.

Wi-Fi works but where I get more than 70 meg with Windows 10, I get less than 10 meg with FreeBSD.

The sleep/wake story now is far better than it was a couple of years ago, largely thanks (I believe) to understated work in the https://github.com/FreeBSDDesktop area. Still, the UX is far from simple:

• FreeBSD bug 238499 – emulators/virtualbox-ose: Getting host panic when resuming if VirtualBox virtual machines are running
• https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194727#c49 (failure to sleep; overlapping problems with sound)

– and so on.

https://github.com/helloSystem/hello/issues/37#issuecomment-728654899

… desktop FreeBSD is a niche within a niche and it kind of shows (Again: just being honest here, no attempt to scare anybody off whatsoever!). …

+1

In 2018 the Foundation positioned FreeBSD as a system for a range of everyday activities, most some of which are associated with personal computers:

– whilst the rear of the flyer used phrases such as personal workstations, the words laptop and notebook are notably absent.

Similarly, now at https://www.freebsd.org/ the word desktops is prominent but no mention of portable computers.

[probonopd]

Honestly I always prefer to work at a "personal workstation" (great name for what I usually call a "desktop computer") with wired networking whenever possible. But yes, the points you mention should be improved; and one way to get them improved is make more FreeBSD developers use it on their desktops. Maybe we can help with that? ;-)

[grahamperrin]

https://github.com/helloSystem/hello/issues/37#issuecomment-728340185

… (rc.d) seems pretty OK in this regard to me. (Will it stay this way when parallel boot capabilities are added?) …

No difficulties so far.

[probonopd]

Are you saying you have a system that already has the new-fangled parallel boot capabilities? (I don't know yet whether and when this will land in anything that we can use for helloSystem, such as the 12.x and/or 13.x series.)

[grahamperrin]

Yes.

[probonopd]

Interesting @grahamperrin. Do you know, how many seconds does a boot need without and with parllel enabled?

[grahamperrin]

I haven't properly compared, I added it in isolation, it'll make more sense to compare after I have updated the system to a version with which it properly belongs. If I find anything interesting I'll post to freebsd-current

[kettle-7]

A good part of that is the kernel taking much longer

The biggest shocker to me is that FreeBSD waits a minute and a half on every boot in case you plug in a USB or CD with a UFS filesystem on it -- why can't we specify the root filesystem in loader.conf or something?

[probonopd]

Sounds like a bug to me. Have not observed this.

[davidchisnall]

@linuxkettle, please file a FreeBSD bug about this with as much detail as you can. The slowest FreeBSD system that I have is an AMD E-350 (dual-core, 1.6GHz, slow microarchitecture). It takes a long time in the BIOS but under 10 seconds from loader finishing to the login prompt. The majority of the post-BIOS boot process is the bit where loader presents the boot menu and waits for a timeout before loading the kernel (you can reduce or eliminate this with a tunable, but I don't because I reboot sufficiently infrequently that a small cost every reboot is worth it for ease of recovery if something goes catastrophically wrong).

[kettle-7]

Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0

It doesn't happen if I put a CD in the drive, so it might be hardware related.

[davidchisnall]

Please file a bug report. I suspect this is a bug in the USB cd driver. Some USB CD drives don't report ready unless there's a disk in the drive. Eventually the driver gives up and reports the drive as not-ready. I suspect there's a longer timeout than necessary here. It may be that the timeout is in something prodding the driver during boot.

Some of these timeouts were introduced for old RAID systems that spun up drives one at a time to avoid overloading the power supply: if you've got a 32-disk spinning-rust array, spinning up all of the disks at once is a huge power draw (a lot more than steady state where they're all spinning with just a little power from the motor to compensate for drag), so a bunch of these arrays would spin up each drive in turn. If it takes 5 seconds to spin up each disk, then a 32-disk array will take 2.5 minutes (in practice you'd probably spin up 2-4 disks at a time, so it wouldn't be that bad). If you had fewer disks, you'd never hit this timeout because the disks would report ready much sooner, so you didn't care. If you had a lot, you didn't care because at least you got to boot in the end. If you apply the same heuristic to a CD drive that never reports ready in the media-not-present state, then you have very slow boot.

[kettle-7]

It's the laptop's built-in CD/DVD drive. With 13.0-RELEASE, the timeout is reduced to 30 seconds, but that's still long enough to do the rest of the boot - in other words, that timeout takes up half of the boot process. It also changes to usbus1.

[probonopd]

Please file a bug report at https://bugs.freebsd.org/bugzilla/ @linuxkettle, thanks.

[kettle-7]

Okay, I've done that now.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256395

[probonopd]

It would not hurt to put the link to a Hardware Probe report into the ticket :-)

[kettle-7]

Looking over old comments.

Linux has Systemd (which merits a lot of critique but regarding service supervision it's definitely some form of progress!)

Yes, systemd in my opinion is one of the best things happening to linux, it makes everything simpler and more efficient.

Also, I think your calculator package @probonopd would be a very good model for how a flatpak-like containerisation system could work, basically having those packages containing a tar archive with the app files to overlay, then a /Library directory containing dependencies that can be overlaid, so that you don't have the flatpak runtime dilemma, then just clean up unused ones on boot. I might implement it if I can get helloSystem working again this weekend.

[kettle-7]

Then there's also haiku, with nearly everything in the base OS, including desktop environment, mounted read only or just referenced in code by the kernel, and then packages that get overlaid. I think if the UI/UX was less outdated and, well, yellow, then it could get up there with linux, because it has security like no other OS despite logging you into the root account, though maybe a password dialog might be good... It's also very intuitive and feels nice to use. Sadly gtk doesn't work yet, otherwise I'd use it as my daily OS.

[probonopd]

Yes. I really like Haiku but I didn't base helloSystem on it because

• There is no graphics acceleration
• They don't want multiple distributions/desktops. They are focused on creating today's BeOS experience. Which is a great goal that I totally respect (but I want to create today's Mac-like experience - that modern Apple has lost imho)
• It doesn't use Qt as the native tookit but something "byzantine" from BeOS times

[kettle-7]

On your first point: yeah, Haiku doesn't support much hardware yet, but that should improve with time. Second: yes, that's one thing with haiku, the desktop environment is part of the kernel. Everything is integrated. Which also means you need to change a lot of kernel stuff to change the desktop. Third: Qt does work, just not gtk. And yes, I know gtk is unwelcome and not to be used for part of the base system but the reality is not every app has been ported to qt.

[kettle-7]

So to conclude the second point is what's holding us back.

[kettle-7]

macOS and Windows have a clearly defined set of desktop-oriented libraries/frameworks that you can assume to "always be there". This does not exist on Linux or BSD.

Yes, there are things you can expect to always be there but they are far and few. Windows and macOS guarantee a widget framework, high level libraries for doing lots of stuff etc.

[jordanhubbard]

Here's a thought - when we were playing around with NeXTBSD (the name was a joke but some people apparently took it seriously?) we did port Mach IPC along with Grand Central Dispatch and some other nice bits of "glue code" that make Mac applications easier to port. If you're looking for a systemd equivalent that is already in wide use, launchd is a reasonable alternative and, as I noted, we already made it much easier to port by bringing the trusted IPC mechanism in that it relies on to ensure that session privileges are genuine.

[kettle-7]

Sounds great!

[probonopd]

Indeed. (Describing launchd as a "systemd equivalent" is quite the understatement, systemd was clearly modeled after launchd.)

For helloSystem, trying to implement this presents a dilemma: We want to stay as close as possible a "true FreeBSD" (e.g., no self-compiled kernel, no self-compiled ports but the stock FreeBSD ones - for very pragmatic reasons) but FreeBSD hasn't implemented the "NeXTBSD" ideas yet.

[jordanhubbard]

One approach that might be worth trying is to reach out to Robert Watson and ask him how he sees the future of trusted IPC evolving in FreeBSD. Trusted IPC means far more than needing elevated privileges to talk to another application or system service - in fact if you need elevated privs to use it, you've already failed. The fundamental goal is to allow any application or service to talk to any other and be able to prove that it is who it says it is via a cryptographic signature that is attached to every message (AKA Mach IPC security trailers). This is why we brought back Mach IPC - we just couldn't find anything else as low-level that did what we wanted with full kernel support. Robert has been involved in the security space for years, so if he has something better in mind, that should be sussed out.

If not, well, Mach technology has been at the core of FreeBSD since the beginning and is STILL in macOS / iOS since every time we wanted to "throw out Mach ports" because they're not exactly easy to understand, we failed because they Just Work and could be wrapped if one wanted easier semantics, so that is of course what happened (XPC wraps Mach IPC, for example).

[kettle-7]

Yes, the hard thing is that we don't want to deviate too much from FreeBSD but the reality is we probably have to, or at least should, because of the many shortcomings of FreeBSD. This is not to say that we can't use BSD as a base but we might need to change stuff like the init system to make it more suited to desktop / gui users.

[joelewing]

I am curious about why FreeBSD was chosen as a base instead of OpenBSD. Either one is a good choice, but OpenBSD has a nice emphasis on security, code correctness, and documentation :)

[davidchisnall]

I'm obviously biased here, but I think there are a couple of FreeBSD features that are really important here that don't have OpenBSD analogues:

• ZFS. Snapshots, rollback, incremental backup, and so on are really important for a usable system. There was an initial ZFS port to OpenBSD, but I think it's stalled and there's nothing equivalent.
• Sandboxing support. FreeBSD has jails for container-like sandboxing and Capsicum for building privilege-separated things. Capsicum was designed in collaboration with Apple and is explicitly designed to support desktop-style workloads where you have a PowerBox abstraction to provide file descriptors to a sandboxed process in response to user action. OpenBSD has pledge, which has a few advantages (it's easier to adopt for existing software, in particular) but it is much harder to reason about the security properties involved for anything that needs to access a dynamic set of files / network interfaces (with Capsicum it is trivial: you can access only the file descriptors that you are explicitly granted. I've used a bunch of different sandboxing frameworks and I've not found anything else that is as easy to make sure that I really have the security properties that I want.).
• Long-term support. A FreeBSD release has a stable ABI for around five years and rolling updates for third-party things on top of that. OpenBSD tightly couples the base system and ports so there's no guarantee that you can build any version of a third-party package that includes a feature update or a security fix, unless you're using the latest CVS release of the base system (I was burned by this and ended up running having to run OpenBSD HEAD for a while to be able to install security updates of the things that the machine was actually meant to be running).
• Packages. FreeBSD has a much better binary package story and, in particular, has tooling in the form of Poudriere for building package sets and VM / ISO images that contain them.

[probonopd]

I am curious about why FreeBSD was chosen as a base instead of OpenBSD.

I chose it partly because Darwin references it (references other BSDs, too) and partly because there was a GitHub repository that produced a Live ISO with a working Xorg setup. This way I could try out FreeBSD without having to install anything. Later I could just fork it and go from there. Never looked back since.

[grahamperrin]

https://github.com/helloSystem/hello/issues/37#issuecomment-927579852

… we might need to change stuff like the init system …

If I understand correctly … please, don't underestimate the potential burden of maintaining ports to be compatible with a change.

[kettle-7]

User facing apps shouldn't need to deal with init scripts, and even so let's say we make something ourselves we can add support for rc.d scripts, like systemd has System V support.

[jordanhubbard]

One might hope by now that a startup mechanism abstraction layer (API + CLI interface) which supported a variety of backend formats would keep ports / pkgs from needing this kind of intrinsic knowledge. MacPorts implemented this over a decade ago since macOS went through a similar evolutionary process and the implementers wanted portability to other OSes as well.

On Oct 8, 2021, at 17:10, the seventh kettle (I think) @.***> wrote:

 User facing apps shouldn't need to deal with init scripts, and even so let's say we make something ourselves we can add support for rc.d scripts, like systemd has System V support.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[davidchisnall]

Note that there is also interest upstream in an improved init system. I would like something that:

• Is easier to plug other event sources (e.g. pending abrupt shutdown notifications in a cloud) into.
• Is easier to add dynamic event consumers to. devd, for example, provides a sequential-packet socket that you can read from, but only for the event firehose. There's no way of dynamically requesting a single event. For example, a DBUS bridge may want to request notifications on insertion of removable media. Currently, you do this by installing a file in /etc/devd.d to run a program when you want the event.
• Uses the reaper support (see: procctl(2)) to do supervision.
• Supports socket activation.
• Can manage processes in a jail, without needing to run a separate copy inside the jail.
• Can start processes in Capsicum capability mode with specific file descriptors pre-opened.
• Uses UCL for configuration (UCL is already in the base system).
• Unifies various different kinds of event sources (timers, socket connections, other services starting, devd events, and so on).
• Is written in a modern language (modern memory-safe C++ for the core and Lua for everything else would be my choice - both are in the base system already and Sol3 makes it really easy to bridge Lua and C++). Most of the system should be written in a memory-safe language, not C/C++.
• Can consume systemd unit files (I don't really like systemd, but I also don't like forcing people who maintain ports to read the unit files that upstream projects write and translate them into something else).
• Respects POLA and can consume existing RC scripts (and crontabs / inetd.conf if it subsumes that functionality).

Note, this isn't actually describing an init system, it's describing a process supervision system. I don't need it to be pid 1.

If someone wants to work on this, I'd be very happy to help with upstreaming, but it can also live in a port and gradually take responsibility for more of the core system until it's ready.

[grahamperrin]

https://github.com/helloSystem/hello/issues/37#issuecomment-938479540

User facing apps shouldn't need to deal with init scripts, …

Not exactly user-facing, but the first port that came to mind was webcamd:

• https://cgit.freebsd.org/ports/tree/multimedia/webcamd/files/webcamd.in?id=1d6b4b3f9145338f9413b8827d5fdb118ac9be1f#n14 (2010-03-27)
• https://cgit.freebsd.org/ports/tree/multimedia/webcamd/files/webcamd.conf.in

– and so on, and I can't guess how many other ports use rc.d.

https://github.com/helloSystem/hello/issues/37#issuecomment-938522410

… consume existing RC scripts …

If I recall correctly, there exists something that does so but (sorry) I can't recall its name.