search

clearlinux / distribution

Placeholder repository to allow filing of general bugs/issues/etc against the Clear Linux OS for Intel Architecture linux distribution
522 stars 29 forks source link

Building and clr-installing legacy releases having no ISOs #3146

Open c13-github opened 4 months ago

c13-github commented 4 months ago

I would like to evaluate 4.1 Kernel releases for evolutionary deep dive of CL*OS. The current bleeding edge may not be the best place to understand all of its optimizations given my rudimentary nix background.

I do use a 6.6 LTS kernel as my dev base. I'm able to clone CL*OS release I want and installer version of the same era.

I presume very early CLOS builds were not bootstrapped . Based on a read of swupd.go, it needs to guess if we are running off a CLOS system or from a source tree. The clr-installer (1.0.0) I can compile and run however guesses wrong and attempts to swupd remote after TUI starts.

swupd then fails (panics) because the bundles.json is not rsynced. I expect workaround is mix/self-host bundles per "How to Clear" guide.

Additionally, install time go-buildxxxx folder in /tmp is cleaned by clr-installer on failure, so no artifact config.json remains for inspection (when started without -c flag and config file).

*Question/issue is can I still dev build and deploy tui bootable installation to removable storage on my non-CLOS box any early (Kernel 4.1 era) non-ISO supplied release as I've described, and what config+flags suit this?**

My retrodev environment:

Thanks for committing to archiving CL*OS for posterity.

bwarden commented 4 months ago

Have you looked at the clr-installer YAML config syntax and the installer configs we use to generate our published images? You should be able to use a current release of clr-installer to create your own bootstrap image of an arbitrary release of Clear Linux.

c13-github commented 4 months ago

clr-installer latest with a config runs mod in /utils that checks /usr/lib/os-release symlink to verify its running on CL'*'OS and for the presence of /bin/swupd. The installer will silently fail with only prior debug log lines of "The current Clear Linux Version is 0".

The links did not shed light on how one can build a release wthout these 2 criteria but if that's expected than will move to Close.

bwarden commented 4 months ago

Going through history, we last released 4.1 kernels back in 2015. I'm not sure we're going to be able to help you with that part very much. My best idea right now is to boot off of a current server image (preferred .img over .iso) and run the installer manually from there, specifying which version you want to try to install, and directing it to a separate bootable device.

fenrus75 commented 4 months ago

what do you mean by bootstrapped? We've been self hosting./self building from the very first release (release 10) of clear linux... (even though the installer came in later)

swupd can go backwards.. with limits. I don't think you can go backwards that far back in one big step.

On Tue, Jun 25, 2024 at 6:31 PM c13-github @.***> wrote:

I would like to evaluate 4.1 Kernel releases for evolutionary deep dive of CL*OS. I understand this is not necessarily a supported ask but the current bleeding edge may not be the best place to understand it given my rudimentary nix background.

I do use a rolling release 6.6 LTS kernel as my dev base. I'm able to clone CL*OS release I want and installer version of the same era.

I presume very CLOS early buids were not bootstrapped . Based on a read of swupd.go, it needs to guess if we are running off a CLOS system or from a source tree. The clr-installer (1.0.0) I can compile and run however guesses wrong and attempts to swupd remote after TUI starts.

swupd then fails (panics) because the bundles.json is not rsynced. I expect workaround is mix/self-host bundles per "How to Clear".

Question/issue is can I build and deploy tui installation of an early non-ISO release as described, and what config+flags suit this?

My retrodev environment: Alpine latest (diskless), Go x64 latest, 32GB removable storage, 64-bit Intel CPU supporting SSE4.1, VT & AVX512. Dependencies are satisfied to go build and go run clr-installer 1.0.0. from the repo. Thanks for committing to archiving CL*OS for posterity.

— Reply to this email directly, view it on GitHub https://github.com/clearlinux/distribution/issues/3146, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FIZXBA664GFOBAOL5TZJIKVLAVCNFSM6AAAAABJ44O3O2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGM3TGOJXGU3TOMI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

c13-github commented 4 months ago

Going through history, we last released 4.1 kernels back in 2015. I'm not sure we're going to be able to help you with that part very much. My best idea right now is to boot off of a current server image (preferred .img over .iso) and run the installer manually from there, specifying which version you want to try to install, and directing it to a separate bootable device.

Thank you @bwarden for taking the time to look into that. Using a certain amount of faking/mocking (e.g. of /etc/os-release, keymap+locale, connectivity checks in /controller) I was able to get clr-installer-1.0.0. to progress to the removable media preparation stage on a non-CLOS box. The most realistic approach I agree is to do it from CLOS with a functional swupd.

The learning exercise was fruitful and the hints you provided actually did turn out to be helpful to that end.

I was expecting zero interest/guidance from the team on an issue in such a niche area (retro kernel building of a rolling distro) but the genuine feedback effort does make me feel I made the right choice in distro to study. I will move to close once I request one more tidbit of information sought re. early CL*OS self-builds and swupd scoping from @fenrus75's comment.

c13-github commented 4 months ago

what do you mean by bootstrapped? We've been self hosting./self building from the very first release (release 10) of clear linux... (even though the installer came in later) swupd can go backwards.. with limits. I don't think you can go backwards that far back in one big step.

Thank you for the knowledge transfer. I did successfully go build clr-installer-1.0.0 and was able to run its tests and even progress to the removable media preparation stage from a non-CLOS box. This is what I was referring bootstrapping but that isn't the term a kernel contributor/developer would likely use for that so likely a misunderstanding.

swupd being the critical component to create CLOS from bundles/sources is the gap that anyone attempting to build a retro CLOS version (who isn't already running CLOS would face).

Has it ever been determined how many CLOS releases one could practically cross? The existence of tools like Fedora's leapp suggests to me that concept is an active topic in enterprise migration scenarios. swupd needing "guessing" code to know its host (e.g. already on CL*OS or from source tree) to me reflects the legacy of self-build. clr-installer's current checks (e.g /etc/os-release, existence of swupd) suggest current development is fully CLOS driven.

If you have more shareable experience/insight into how 1.0 was self/host & built (e.g. tooling/OS), those in the retro kernel building community could likely learn a great deal from it. I tried searching for archived 01.org content re. CLROS 1.0 on the Internet Archive, but it seems much was never captured. I feel it would be a shame if future posterior tinkerers and would-be contributors would never know that piece of CLOS history.

bryteise commented 4 months ago

Has it ever been determined how many CLOS releases one could practically cross? The existence of tools like Fedora's leapp suggests to me that concept is an active topic in enterprise migration scenarios. swupd needing "guessing" code to know its host (e.g. already on CL*OS or from source tree) to me reflects the legacy of self-build. clr-installer's current checks (e.g /etc/os-release, existence of swupd) suggest current development is fully CLOS driven.

In swupd's case the question is more about formats, basically we have a file that swupd uses to know about OS contents and when the file format's change then swupd will fail. We get around this by making a "format bump" release. Two builds, say version 10 and 20 are made where the OS contents are the same but 10 will have the old file format and 20 will have the new file format. Doing an update this way, swupd will get version 10 but it will actually be version 20 once the update is complete.

Going forward this works nicely because once the update is complete the new swupd on the system will know about the new format in 20. Going backwards however does not work. We haven't done many format bumps that actually will break though but it is hard to know when those happened without looking at each format bump.

If you have more shareable experience/insight into how 1.0 was self/host & built (e.g. tooling/OS), those in the retro kernel building community could likely learn a great deal from it. I tried searching for archived 01.org content re. CLROS 1.0 on the Internet Archive, but it seems much was never captured. I feel it would be a shame if future posterior tinkerers and would-be contributors would never know that piece of CLOS history.

I think the easiest way to "build" the OS content wise would be to take the rpm repo, https://download.clearlinux.org/releases/2580/clear/x86_64/os/repodata/ for instance has a 4.1 kernel in it. You'd not be using swupd but if you configure dnf to use that repository you could dnf install --root to build a clear linux system into a chroot which you could make a disk image out of.

You'd need to manually setup a bootloader and configure it but with the right packages you'd be able to construct something bootable.