search

jhuntwork / merelinux

A lightweight Linux distribution using musl libc, pacman and s6
MIT License
87 stars 8 forks source link

Runs on bare metal? #50

Closed subnut closed 3 years ago

subnut commented 3 years ago

Is it possible to run this on bare metal? More specifically, a Thinkpad E495.

Thank god I found this... I was just going to make this EXACT SAME THING myself! standards of musl + reliability of s6 + familiarity of pacman, a heavenly choice!

jhuntwork commented 3 years ago

Hi @subnut thanks for asking! Sorry for the late reply. Technically, yes it is able to run on bare metal, I have a personal system that is currently using it. However, I haven't yet done much to generalize the kernel for other systems. You would likely have to build your own custom kernel at this point.

After a bit of a hiatus, I'm looking at providing some updates as well as making this more generally usable. Stay tuned for more updates, I'll try to have a game plan mapped out soon.

subnut commented 3 years ago

Great to hear that!

https://github.com/jhuntwork/merelinux/blob/72ecc8befcfb5d0a05368d22b714fb52a2901f56/docs/index.html#L184

So... I guess we shall be using statically-linked toybox/busybox/sbase/ubase instead of GNU coreutils?

jhuntwork commented 3 years ago

Yes, I prefer static binaries where possible/reasonable. And I prefer avoiding GNU where reasonable as well. It's difficult to completely remove GNU utils, and there's no doubt lots of cases where doing so would be extremely uncomfortable. The base system uses busybox by default for now (toybox was incomplete when I began) and so far I haven't seen too many cases where busybox isn't usable.

fungilife commented 3 years ago

I think Glaucus linux is using toybox http://glaucuslinux.org/

jhuntwork commented 3 years ago

I think Glaucus linux is using toybox http://glaucuslinux.org/

Nice. I've been watching the progress of toybox for a long time. We could take another look at it, but the reason I chose busybox and have stuck with it up to now is that it had more features that allowed me skip several external packages while maintaining a small, simple size. As just a few examples: awk, gzip, sh, a dhcp client (udhcpc)

I'd be interested in hearing other's thoughts about it, though.

jhuntwork commented 3 years ago

@subnut do you know what modules your Thinkpad uses?

subnut commented 3 years ago

@subnut do you know what modules your Thinkpad uses?

Umm, no..... How to find that out?

jhuntwork commented 3 years ago

If you are running another linux system on it already (or you can launch linux from an iso or usb) you should be able to run lsmod from a terminal

subnut commented 3 years ago 
$ lsmod
Module                  Size  Used by
ccm                    20480  6
algif_aead             16384  0
cbc                    16384  0
des_generic            16384  0
libdes                 24576  1 des_generic
ecb                    16384  0
8021q                  40960  0
garp                   16384  1 8021q
mrp                    20480  1 8021q
algif_skcipher         16384  0
stp                    16384  1 garp
llc                    16384  2 stp,garp
cmac                   16384  0
md4                    16384  0
algif_hash             16384  0
af_alg                 32768  3 algif_hash,algif_skcipher,algif_aead
uinput                 20480  1
vfat                   24576  1
fat                    86016  1 vfat
joydev                 28672  0
mousedev               24576  0
uvcvideo              118784  0
videobuf2_vmalloc      20480  1 uvcvideo
videobuf2_memops       20480  1 videobuf2_vmalloc
videobuf2_v4l2         36864  1 uvcvideo
videobuf2_common       69632  4 videobuf2_vmalloc,videobuf2_v4l2,uvcvideo,videobuf2_memops
videodev              282624  3 videobuf2_v4l2,uvcvideo,videobuf2_common
mc                     65536  4 videodev,videobuf2_v4l2,uvcvideo,videobuf2_common
intel_rapl_msr         20480  0
intel_rapl_common      28672  1 intel_rapl_msr
edac_mce_amd           32768  0
wmi_bmof               16384  0
kvm_amd               139264  0
amdgpu               7053312  13
kvm                  1036288  1 kvm_amd
iwlmvm                495616  0
irqbypass              16384  1 kvm
gpu_sched              45056  1 amdgpu
i2c_algo_bit           16384  1 amdgpu
crct10dif_pclmul       16384  1
crc32_pclmul           16384  0
mac80211             1171456  1 iwlmvm
drm_ttm_helper         16384  1 amdgpu
ghash_clmulni_intel    16384  0
ttm                    77824  2 amdgpu,drm_ttm_helper
snd_ctl_led            24576  0
aesni_intel           380928  4
drm_kms_helper        294912  1 amdgpu
snd_hda_codec_conexant    28672  1
crypto_simd            16384  1 aesni_intel
cryptd                 28672  2 crypto_simd,ghash_clmulni_intel
cec                    73728  1 drm_kms_helper
libarc4                16384  1 mac80211
rapl                   16384  0
snd_hda_codec_generic    98304  1 snd_hda_codec_conexant
snd_hda_codec_hdmi     73728  1
iwlwifi               430080  1 iwlmvm
snd_hda_intel          57344  0
snd_intel_dspcfg       28672  1 snd_hda_intel
snd_intel_sdw_acpi     20480  1 snd_intel_dspcfg
psmouse               184320  0
pcspkr                 16384  0
acpi_call              16384  0
k10temp                16384  0
sp5100_tco             20480  0
tpm_crb                20480  0
snd_hda_codec         176128  4 snd_hda_codec_generic,snd_hda_codec_conexant,snd_hda_codec_hdmi,snd_hda_intel
cfg80211             1044480  3 iwlmvm,iwlwifi,mac80211
i2c_piix4              28672  0
drm                   585728  10 gpu_sched,drm_kms_helper,amdgpu,drm_ttm_helper,ttm
r8169                  98304  0
snd_hda_core          110592  5 snd_hda_codec_generic,snd_hda_codec_conexant,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec
realtek                28672  1
agpgart                45056  2 ttm,drm
snd_hwdep              16384  1 snd_hda_codec
mdio_devres            16384  1 r8169
syscopyarea            16384  1 drm_kms_helper
thinkpad_acpi         131072  0
snd_pcm               147456  4 snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec,snd_hda_core
sysfillrect            16384  1 drm_kms_helper
platform_profile       16384  1 thinkpad_acpi
snd_rn_pci_acp3x       20480  0
sysimgblt              16384  1 drm_kms_helper
ledtrig_audio          16384  3 snd_ctl_led,snd_hda_codec_generic,thinkpad_acpi
tpm_tis                16384  0
snd_pci_acp3x          20480  0
snd_timer              45056  1 snd_pcm
ccp                   118784  1 kvm_amd
libphy                151552  3 r8169,mdio_devres,realtek
fb_sys_fops            16384  1 drm_kms_helper
rfkill                 32768  3 thinkpad_acpi,cfg80211
tpm_tis_core           28672  1 tpm_tis
ucsi_acpi              16384  0
snd                   114688  10 snd_ctl_led,snd_hda_codec_generic,snd_hda_codec_conexant,snd_hda_codec_hdmi,snd_hwdep,snd_hda_intel,snd_hda_codec,snd_timer,thinkpad_acpi,snd_pcm
typec_ucsi             49152  1 ucsi_acpi
tpm                    90112  3 tpm_tis,tpm_crb,tpm_tis_core
soundcore              16384  2 snd_ctl_led,snd
typec                  65536  1 typec_ucsi
roles                  16384  1 typec_ucsi
rng_core               16384  2 ccp,tpm
wmi                    36864  1 wmi_bmof
i2c_scmi               20480  0
video                  57344  1 thinkpad_acpi
pinctrl_amd            32768  0
acpi_cpufreq           32768  0
mac_hid                16384  0
ext4                  929792  2
crc32c_generic         16384  0
crc16                  16384  1 ext4
mbcache                16384  1 ext4
jbd2                  151552  1 ext4
rtsx_pci_sdmmc         32768  0
serio_raw              20480  0
mmc_core              188416  1 rtsx_pci_sdmmc
atkbd                  36864  0
libps2                 20480  2 atkbd,psmouse
xhci_pci               20480  0
i8042                  32768  0
crc32c_intel           24576  4
rtsx_pci              106496  1 rtsx_pci_sdmmc
xhci_pci_renesas       20480  1 xhci_pci
serio                  28672  7 serio_raw,atkbd,psmouse,i8042
jhuntwork commented 3 years ago

Looks like you have the r8169 network interface and the amdgpu, those are probably the most significant ones. I have those built as modules now, but am still missing some of the others. For example, I haven't started looking at wifi yet.

As it stands right now, the machine will probably boot correctly with mere's linux package in the testing repo, but some pieces, like wifi wouldn't work. Network over your ethernet port should work fine though.

fungilife commented 3 years ago

I have been experimenting some with an arch-based no-systemd system, Obarun in specific, to see how life would be without udev. I have employed smdev, nldev, and libudev-zero. The first two have been idling around for years, the later is ongoing development as we write/read. My window manager has 100% functionality with it, very few applications threw out an error. So, if you can leave some udev setup/hooks to be easily substituted. https://github.com/illiliti/libudev-zero/issues/4

The only source of headache for some arch sw is the reliance to libudev providing version information, and the key to this disfunctionality revolves around lvm2/device-mapper. Substituting nldev for udev in mkinitcpio was much easier than I had thought, booted fine first try.

Taking the advice from the thread above I was able to build lvm2/device mapper udev free, I really thought I would be dealing with days if not hours of debugging and patching. It is hard to test everything against it since it is crawling everywhere, but it seems functional. The next phase would be substituting smdev/nldev for mdevd from skarnet, which is a little tougher to do, but the two projects libudev-zero/mdevd are in communication collaboration it seems.

I am watching silently your explosion of work, I am waiting for the dust to settle a bit before giving it a try again since a year ago, but it is very inspiring. This is heading for true freedom, unlike fedora, debian, arch,.. etc heading for being standardized IBM-RH forks.

fungilife commented 3 years ago

I don't get this, your PKGBUILD for pacman appears to be making makepkg which you rename pacman-build, for unknown reasons, yet the pacman you have in the repository doesn't include this package.

What do we do? Somehow copy the config from you PKGBUILD and attempt make install of the real package? I think it is very poor practice to show as source a PKGBUILD that doesn't correspond to the binary you offer in the repository.

I was under the very poor assumption that the pacman upgrade you did incorporated pacman in its entirety. This is useless.

jhuntwork commented 3 years ago

The pacman PKGBUILD creates two packages, or split packages, one called pacman which is run-time only requirements for systems that aren't planning on developing packages and has no dependencies, and pacman-build which has makepkg and friends, other developer tools that ship with the pacman source. To build packages, you need pacman-build. Sorry if that causes confusion.

I'm happy to consider alternative suggestions. The main objective in splitting them that way was to keep the base system footprint small for systems and use cases that don't plan on building packages.

jhuntwork commented 3 years ago

More details about the package splitting feature: https://archlinux.org/pacman/PKGBUILD.5.html#_package_splitting

fungilife commented 3 years ago

I came back to edit my venting frustration, I figured it out a little later. I started building nano thinking it is simple enough to get accustomed. Using an arch-base PKGBUILD makepkg needs to skip gpg checks: makepkg --skippgpcheck

I see makepkg.conf is quite different from what arch is using.

I get an errror (after many others trying to figure out what install flags work and don't with busybox:

==> ERROR: Missing dependencies: ld-musl-x86_64.so.1

But the pkg is really all there and correct in ./pkg/nano/.. but doesn't get transfered to a tar ball I tried to fool the system linking what is looking for back to /lib/ld...so but no difference.

Also, before even checking, I assumed the system needed a user to build packages, as it is common in pacman based distros, that ONLY user can build, only root can install. It seems you canged this configuration. Then while working on chroot /tmp is not a tmpfs and user couldn't write to it, and it seems as makepkg is trying to send the tarball to /tmp/staging but using root to build didn't produce anything either. Same error. Luckily just copying /usr/bin/nano and /etc/nanorc worked fine, so I am happy to have my favorite editor going.

If I can get the ball rolling escaping those errors I can add a repository here and share my successful pkgbuilds

What is the correct and complete PATH for a sh in Mere ...
There was some utility I was missing and figured it would be in binutils, I didn't realize it existed elsewhere (maybe it was strip) and binutils was installed in /opt only opt is not in the standard path so it needs an absolute path. Then I realized that some parts of binutils and llvm are in conflict ...

I can help with documentation but I need to find my way through it first.

fungilife commented 3 years ago

I didn't realize the splits before, I am in favor of it, and I was used to it in void, they sometimes split in 3 having the documentation also optionally installed.

jhuntwork commented 3 years ago

I can help with documentation but I need to find my way through it first.

Thanks for not giving up! 😄 Yes, there is quite a bit that is different from Arch, and I started documenting what needs to be done to for developmet, but it really needs a lot more details.

A few things to keep in mind:

There's more, but I don't have a lot of time right now. I'll try to make some more decent walkthroughs of a typical workflow when I have a moment, perhaps later this evening.

The oddity about /tmp is funny, it should be set to 1777 so that every user can write to it, even if it isn't a tmpfs. But am also happy to consider turning it into a tmpfs as well.

fungilife commented 3 years ago

I tried a couple more pkgs and I get the same error about missing dependency ==> ERROR: Missing dependencies: ld-musl-x86_64.so.1

I did manually change /tmp to 777 so both user and root can write to it

jhuntwork commented 3 years ago

The missing dependency error is just the dependencies.sh script telling you that you need to define that dependency in the package itself. It found it because your build produced a dynamic binary or library.

To be able to package successfully, you just need to add 'ld-musl-x86_64.so.1' or, maybe better, "ld-musl-$(arch).so.1" to a depends variable in the PKGBUILD file. See this one for an example: https://github.com/jhuntwork/merelinux/blob/main/packages/sudo/PKGBUILD#L49-L51

I added that feature in because I kept getting caught by hidden library dependencies that were hard to track manually.

fungilife commented 3 years ago

Thank you, I've certainly made some progress with this fix, I was under the impression that musl alone would take care of this but it needs the specific library link to be happy.

On another note some pkgs I am trying to build fail when they involve a test/check at the end of the building https://termbin.com/vpxm

The behavior is similar, no output indicating specific problem, no process other than makepkg hanging on, but it stays there like this and does not proceed, even after I have forgotten what I was doing :)

Disabling checks/tests wouldn't be a good idea.

In the particular example I've been trying to build zsh basing the attempt on arch's zsh pkgbuild, which requires libcap and pam, gdbm (I've built that one), and pam needs pambase and a ton of other things that don't exist yet. So I tried building with disabling pam and cap at the configuration.

But then this happened and reminded me of a couple of other pkgs that hang in the same part of the building.

Any clues on what to do would be greatly appreciated.

jhuntwork commented 3 years ago

Do you mind uploading the full PKGBUILD files somewhere, maybe gdbm and zsh?

I think libcap should be ok to add too, but pam might be a challenge. At least, the last I looked at it, it was a little hairy.

fungilife commented 3 years ago

I sent an email too https://framagit.org/mere

Whatever I work on, will go here, if it is not working I will indicate it on the README of each pkg,

gdbm and nano work, libcap zsh aren't.

jhuntwork commented 3 years ago

Looks like unsetting CFLAGS will keep it from hanging. It's either the difference between -O2 and -Os in clang, or more likely, that we're missing -fPIC. I'll dig a little further. In the meantime, alpine disables that test anyway, for an unrelated failure: https://github.com/zsh-users/zsh-docker/issues/5#issuecomment-320077830

jhuntwork commented 3 years ago

This is really surprising, but I've narrowed it down to having -Werror-implicit-function-declaration set. Having CFLAGS just set to that will cause the tests to hang. It doesn't seem like these should be connected...

jhuntwork commented 3 years ago

Some of the folks in the #musl channel on LiberaChat helped track it down to autoconf settings. If zsh's configure system doesn't include the right headers in their checks, some of configure's tests will fail resulting in a misconfigured build. It looks like there's an upstream fix for it here: https://github.com/zsh-users/zsh/commit/bd647c156549b2f666e5fae80f1ca674b6cde895 although it's unreleased. That doesn't apply cleanly to the 5.8 package though.

Separately, in their README, there's this note that will probably be helpful:

Build-time change: The default value of the --enable-gdbm configure argument has changed from "yes" to "no". Thus, the zsh/db/gdbm module will not be built unless --enable-gdbm is passed explicitly.

jhuntwork commented 3 years ago

I'm going to close this issue now as the main question should be answered through the INSTALLING document now. Of course there are still more features to add to make the answer to this question really nice, including more hardware support and a desktop environment, but we'll work on those in separate issues.

@fungilife feel free to update here if you like, or maybe add a new issue or discussion around the packages you're working on.