puppylinux-woof-CE / woof-CE

woof - the Puppy builder
GNU General Public License v2.0
393 stars 282 forks source link

chroot and a simplified processing of packages #1364

Closed wdlkmpx closed 4 years ago

wdlkmpx commented 5 years ago

I've been slowly testing and updating the code that resides in woof-next.

It uses chroot and a simplified method to process packages, i like it, several times faster than the current stuff, but it doesn't use pet packages, hehe... It's meant to build pups that use the compat distro package manager.

So far i've been only testing and updating the woof-next slackware stuff, the deb-build stuff is probably completely broken.

The current method in testing is handy for cross-builds between two different architectures, between puppies from 2 different compat distros.. but it's not that good if not doing a cross-build.

The current stuff is good for cross-builds, that's for sure, but a new alternate method should be implemented as part of the current build system based mostly on what i see in the woof-next branch.

Many hacks and packages-templates are still needed, well, at least a bunch of hacks.. an amalgamation of hacks could be applied after all the packages have been processed.

Four possible SFS's

main.sfs devx.sfs doc.sfs nls.sfs

The need for chroot: so that post-installation scripts work properly,

"get rid of the multiarch symlink" should be merged and proper handling of deb packages should be implemented. etc.

These are just ideas, it's not hard for me to implement them, if time permits of course

The problem is dealing with somebody else's builds and stuff, this could happen in stages..

So this is something that could happen in the future. If time permits.

jamesbond3142 commented 5 years ago

On Fri, 29 Mar 2019 20:33:58 +0000 (UTC) wdlkmpx notifications@github.com wrote:

I've been slowly testing and updating the code that resides in woof-next.

It uses chroot and a simplified method to process packages, i like it, several times faster than the current stuff

I'm glad that you like it. I hope you enjoy reviewing / working with the code as I enjoyed when I wrote it :)

I tried to be as clean as possible, so if you have any questions, just ask. I may not be able to give immediate answer as it was from 5 years ago, but I'm sure sooner or later it will come to me, if it's really needed.

but it doesn't use pet packages, hehe... It's meant to build pups that use the compat distro package manager.

Just like you have intense dislike for ROX, I have intense dislike for PPM :)

So far i've been only testing and updating the woof-next slackware stuff, the deb-build stuff is probably completely broken.

Probably. Debian sid will be broken because Debian changed a lot over the last 5 years (and "sid" is basically the "leading edge"), e.g. systemd is just one of them.

The Ubuntu build, if Ubuntu still keeps its repo for Trusty, should still work. But I'm not sure if it's the case.

The current method in testing is handy for cross-builds between two different architectures, between puppies from 2 different compat distros.. but it's not that good if not doing a cross-build.

The current stuff is good for cross-builds, that's for sure, but a new alternate method should be implemented as part of the current build system based mostly on what i see in the woof-next branch.

Feel free to pick and choose stuff from woof-next, be it ideas, codes, etc.

Many hacks and packages-templates are still needed, well, at least a bunch of hacks.. an amalgamation of hacks could be applied after all the packages have been processed.

True. Woof-CE is basically two-in-one:

  1. It's a Puppy builder.
  2. It's Puppy base code, which is the "essence" of Puppy.

Woof-next only fixes (1); (2) is supposed to be fixed by the actual puppy builder using particular compat distro, and contribute it back to me for merging. But (2) never happened.

The current snapshot of (2) in woof-next was only good enough to boot to the desktop, but I didn't test beyond that. I was careful not to make too many changes because I wanted to track (2) with upstream (=testing), so all those extra hacks went into a separate package, which lives in lightofdawn.org/puppy-extra, if you want that I can send you those packages (the builders would have downloaded them anyway).

Enjoy.

-- James B

wdlkmpx commented 5 years ago

I must admit i'm not that interested in debian and derivatives, so i'll test deb-build.sh sometime in the future.

I was to able make slackware 14.1 boot to desktop and use the mouse.

Now i'm using slackware 14.2 as the base, it boots... to console. 14.2 requires many new dependencies.

I added this repo http://ponce.cc/slackware/slackware-14.2/lxde/

But i still haven't specified new packages

Well the current stuff in testing is heavily patched to be more compatible with slackware/debian by default and i have woof-next already integrated into testing as a separate build system that uses all the current stuff and calls a couple of helpers in woof-code/support

slack-build.sh ./woof-code/support/rootfs-hacks.sh $CHROOT_DIR chroot $CHROOT_DIR /etc/rc.d/rc.update w

build-iso.sh ( cd initrd-progs rm -f initrd.[gx]z ./build.sh -prebuilt -auto -arch ${WOOF_TARGETARCH:-default} mv -f initrd.[gx]z .. )

So if i don't end up implementing stuff, at least woof-next can be added as a separate build system (a proof of concept) that is integrated into testing..

wdlkmpx commented 5 years ago

The good thing about slackware is that it doesn't really require chroot for many packages.

It follows the same logic as pet packages with post-install scripts for woof.. in fact the current stuff actually produces a proper slackware puppy, but without the package manager.

mv usr/lsls ln -s sss usr/bin

So i see a problem with trying to produce a proper devx with something other than puppy/slackware packages

I'm trying to moludarize 2createpackages and 3builddistro a bit more, so that more code and hacks are shared between the 2 build systems

I'll be pushing all the changes to my woofce fork, just in case i accidentally delete all my work (something i have done before), i'll add the first and trivial changes to woof-CE upstream a few weeks later, after some proper tests.. nothing disruptive in any (disruptive) way

wdlkmpx commented 5 years ago

A year ago, a brutal cleanup happened in woof-distro/ .. it was a dictatorial move, but it was for the better, there's no other way to move things forward..

I did it so that it's easier for people to identify what's wrong in their builds

I believe there are still some entries to delete

This is a fact:

yes|pkg|pkg1[,pkg2|| = compat package yes|pkg|| = pet package

findpkgs and related C app uses a complex logic to find the packages to be included in the build... it will be simplified a bit

I think repo redirections are only for pet packages, i'm not sure maybe woodenshoe-wi has a better recollection .. repo redirections will not be supported..

yes|pkg||pet:repo will be ignored,

The logic will be simplified a bit, so that this doesnt happen again: petcompiledPTNS="${petcompiledPTNS} |${DISTRO_BINARY_COMPAT}| |"

increasingly desperate. 130312 add wary5.

It will findpkgs packages using a simplified logic following the order defined in: $PACKAGELISTS_PET_ORDER - 'PET_REPOS' file

@peabee @mrfricks

wdlkmpx commented 5 years ago

On second thought repo-overrides do seem to apply to compat distro packages,

yes|mozilla-firefox|mozilla-firefox|compat:salix = firefox from the salix repository

For slacko 14.0 it downloads mozilla-firefox-52.9.0esr-i686-1gv.txz

in findpkgs (PKGS_SPECS) you can specify a wildcard for pkgs to be selected, check dependencies, exclude some packages.. all at the same time

I guess that's good, but the the current stuff is quite disjointed, the scripts do their own thing, the same logic doesn't apply everywhere, it's hard to properly think what to do. woof-next doesn't have that problem

The idea was good but the design was fundamentally wrong, everybody's talent went to waste trying to improve and fix something that really required a complete rewrite..

wdlkmpx commented 5 years ago

2createpackages & 3builddistro should become 1 script. 2 become 1.

doc>nls, nls>exe, etc, should be ignored.

The only redirection that should be recognized for backward purposes should be exe>dev and maybe exe>null,dev>null,doc>null,nls>null

If a package is a devx package, it should be: devx, if a package is not wanted, then it should be nullx (fake install). yes|gcc|gcc|devx yes|systemd|systemd|nullx

The script should just process all the packages, whole packages, and then move DEV, DOC, and NLS stuff to their respective SFS's.

This approach allows two different methods to work more or less properly, "native" builds and cross-builds. And more importanly 3builddistro doesn't need to guess what packages will be in the build. 2 become 1.

Fixing stuff requires a redesign, that's not a secret, but the problem is.. everybody else.

I'm writing this through the fever

Any thoughts?