Improving puppy modularity

[rizalmart]

Puppy today can be modularized by the following Puppy SFS - Main puppy rootfs ZDRV - Kernel module layered mountpoint at /lib/modules FDRV - Firmware drivers layered mountpoint at /lib/firmware

YDRV - Extras layered mountpoint at /opt

ADRV - Application suites.

The problem here was ADRV unlike ZDRV, FDRV, and YDRV which the content was isolated in a directory. ADRV was scattered everywhere on the root filesystem. Think makes puppy difficult to organize its modularity. I suggest that ADRV contents must put on /usr/local since the contents there were few.

[sc0ttj]

I like the idea of this...

I do like the idea of at least somewhat following the convention of having

• "base" packages at /bin and /usr/bin
• user installed packages at /usr/local/bin, /usr/local/share/ (etc)

However, we don't have control over the contents of the Debian/Ubuntu/Slackware packages, which also seem not to follow this convention.

So while I like the idea, is it even achievable for more than a few self-produced packages? (genuine question)

Cos I don't think it's worth adding code and complexity to any existing Puppy tooling or system scripts that would then manually fudges things into /usr/local (unless it's way easier than I think - but there will be libs shared by programs in "base" and "user" packages, for example.. niggles like that.. )...

[peabee]

I am against this idea / proposal. The current implementation is flexible and does not need to be constrained in any additional ways. The only thing defined about the existing sfs's is the order in which they are layered - their content can be anything although the z and f do conventionally contain the kernel and firmware - but they don't have to and thats good. 64-bit systems have much bigger problems in that every distro seems to have a different 64-bit lib model.

[rizalmart]

@peabee do you have any ideas on how to make ADRV modular? The remaster script right now can create FDRV aside from ZDRV. Adding create ADRV on remaster script will be a big bonus

[s243a]

@peabee do you have any ideas on how to make ADRV modular? The remaster script right now can create FDRV aside from ZDRV. Adding create ADRV on remaster script will be a big bonus

By default, all changes that aren't part of the FDRV and ZDRV could be placed into the ADRV unless they modify a file in the base sfs file. One could do this by checking to see if a file that changed is part of the basesfs file.

There could be a mechanism to place a selected amount of packages (and dependencies) into the basesfs file.

[peabee]

Hi @rizalmart / mistfire & @s243a Woof-CE already has features to make ADRIVE YDRIVE & FDRIVE in _00build.conf but my adrv predates them and I don't use them - I hand construct the adrv, place it in sandbox3 and have code in EXTRA_COMMANDS to move the adrv into the build. If I were to remaster (which I never do) I would probably remaster WITHOUT the adrv being loaded. There is a danger in trying to push remaster too far I think. isos should just be the base system and any add-ons should be separately distributed as additional sfs (ydrvs or adrvs or specific sfs) - e.g. see my ydrvs for LXDE and LXQt and my sfs for browsers.

[gyrog]

In Puppy Linux, .sfs files contain the files in the appropriate directories, to make them "fit for purpose". When an .sfs file's mount point is added to the aufs stack as a branch, the files simply appear in the directories where they are needed. So binaries are already in appropriate directories, libs are already in appropriate directories, ".desktop" files are in the appropriate directory, etc... When the .sfs file is removed from the stack, the files simply disappear from the root filesysem, (which is the aufs stack).

The adrv...sfs is one of these.

[rizalmart]

Another problem with SFS modules was PPM and Pkg unable to see those packages/components loaded by SFS-modules.

In TazPuppy solves this by making the installed packages database dynamic. Since tazpkg creates folders of a packages entry it can easily rebuild installed package database.

SFS modules created by TazPuppy has this folders like this inside SFS modules

/var/lib/tazpkg/installed/foo1

The main SFS of TazPuppy has installed folder contains like this

/var/lib/tazpkg/installed/foo2 /var/lib/tazpkg/installed/foo3

When SFS loaded is loaded on Puppy it will look like this

/var/lib/tazpkg/installed/foo1 /var/lib/tazpkg/installed/foo2 /var/lib/tazpkg/installed/foo3

Then a installed package will be generated upon load and unload SFS

To implement like this on Puppy. PPM will saved package entry in a specs file in /var/packages/package-specs/

This will be able to generate /var/packages/layes-installed-packages dynamically using this command

cat /var/packages/package-specs/*.specs > /var/packages/layers-installed-packages

This will occur if SFS file was loaded/unloaded.

Newly SFS files will now have specs files that will act as installed packages entry. Not just a single specs file but multiple specs file which SFS module was made

This makes puppy even more modular on SFS modules act like pieces of puzzle.

[peabee]

Assuming we are talking about frugal installs and NOT full installs.... PPM has nothing to do with SFS.... Packages (e.g. .pets) place items into the savearea which may subsequently have to be removed if the package is no longer needed. SFS don't pollute the savearea (other than for any dynamic config items the app needs) - they can be simply removed just by removing the SFS so all the complication of PPM is unecessary and un-needed. There is no need for any changes.

[sc0ttj]

There is no need for any changes.

I agree with peabee... But for completely different reasons..

Another problem with SFS modules was PPM and Pkg unable to see those packages/components loaded by SFS-modules.

This is not quite correct.

Pkg actually can process SFS files as normal packages, and know their contents, and if they are installed or not..

But you have to create the SFS using Pkg for best results..

I plan to improve it a bit, buts it's more or less working fine.. IIRC..

(Pkg creates SFS packages with a pinstall.sh and a ~/.packages/*.files, so they can be treated as normal Pkgs.. It also uses sfs_load to mount/unmount any SFS you try to "install" in the usual ways, like pkg install foo, etc.)

So to be clear.. You can do:

pkg sfs-combine gimp

And you will get one SFS, with all deps bundled in, and all "postinst" file stuffs combined into a single pinstall.sh, called (something like):

gimp_2.38-1-COMBINED-fossapup-amd64.sfs

..and if you install that SFS using Pkg, then it will run its pinstall.sh

---

Newly SFS files will now have specs files that will act as installed packages entry. Not just a single specs file but multiple specs file which SFS module was made

^ in fact, that is what I plan to do with Pkg... Make it build an SFS that contains all the *.files of the packages that went into it, instead of one big, combined one - so the package and all its deps are listed as installed, once the SFS is mounted.

Also doing all this with Pkg instead of the TazPuppy stuff means you don't need to mess around with any other files..

@rizalmart maybe you can simply help improve this feature in Pkg?

NOTE: Pkg also has a merge command, which lets you combine an arbitrary list of packages together into a single one, with the --with-deps option, so you can bundle their deps in too. This should make it (quite) easy to create various SFS files..

can easily rebuild installed package database

FYI, Pkg also has a repack command, which lets you rebuild a PET from anything already installed, by getting its contents from ~/.packages/*.files..

[sc0ttj]

And speaking of ADRV etc... I'm just gonna put it out there: "adrv", "zdrv", "fdrv"? ....Really?!

They should be called something helpful, not cryptic and weird..

Like

puppy-9.5.sfs
kernel-5.4.33.sfs      (?? ..meant be be zdrv)
firmware.sfs
x-puppy-9.5.sfs         (just xinitrc, startx, xlibs, xorg+libs, gtk, PPM/Pkg GUI ... not much else) 

( ^ bear in mind I don't actually know if the zdrv is the actual kernel, or modules for it or what .. but you get the idea)

If x-puppy-9.5.sfs is present, we mount..

..doing it like that also opens up the potential for optional extra SFS files named like:

x-puppy-9.5-jwm.sfs     (jwm, rox, etc) 
x-puppy-9.5-xfce.sfs
x-puppy-9.5-lxde-1.0.sfs
x-puppy-9.5-lxde-1.1.sfs (a fixed on that comes later, easily swapped in)
...

..if x-puppy-9.5.sfs was mounted, we offer to mount one of the x-puppy-[jwm|xfce|ldxe|mate|etc] at boot..

..and then, if using X, we mount all "x-puppy-apps*" that we find...

x-puppy-9.5-apps.sfs           # the "adrv" ... small gtk apps, gnumeric etc
x-puppy-9.5-apps-gnome.sfs           # apps that need gnome libs (...libs included in SFS)
x-puppy-9.5-apps-elementary.sfs    # apps that need elementary libs (...libs included in SFS) 
...

This way people

1. know what each SFS is actually for, at a glance, and without having to trawl through a dozen old forum posts..
2. can get a CLI only puppy by just NOT using any x-puppy* SFS files
3. can swap desktop environments simply by changing an SFS and restart X
4. can update the base system (puppy-9.5.sfs) all on its own, etc
5. can swap and combine different set or collections of desktop apps very easily

That's how you improve puppy modularity.

The benefit of this is that

1. we hardly need to touch the PPM or Pkg, mostly updating the init script, and building the relevant SFS files.. Most would of course be optional extras..
2. sane version numbers and filenames everywhere ... consistent, each file clearly labeled with the pup it's for.
3. If Pkg built the SFS files, then all packages inside those SFS files with have their own ~/.packages/*.files, and will be counted as an installed package - not some file in the main fs that "comes from god knows where"

Bear in mind you can still swap the word "puppy" in all the above examples with stuff like "fossapup", "dpupbuster" and so on, and it still applies/works..

[rizalmart]

@sc0ttj I got your point however on #3 that you said upon inspecting the contents of SFS I noticed that there is no specs file just like in PET packages. Without this PPM will continue to download deps even its already exists because of loaded sfs which consumes more disk space.

I suggest that adding specs files upon creating SFS modules wil help. However PPM reads only one line on pet.specs upon a package installation that's why I suggest to keep a backup copy of specs files in order to generate layer-installed-packages and have an option for PPM to allow use of SFS modules for dependencies.

[CollaboratorGCM]

@sc0ttj , Thanks! Clear file representation in todays world helps speed adoptions.

[sc0ttj]

Clear file representation in todays world helps speed adoptions.

Indeed.. Here's some fun facts:

• 90% of people leave a webpage after just under 2 seconds of waiting for it, according to the best research..
• the "most engaging" (longest avg time being read) articles in UK get 6 mins or so (for 6,000 words, imgs, etc)
• most articles are looking at more like 50 seconds if they're lucky
• people rarely bother to get more than 1/3 of the way down a page

..Here's the point:

If we wanna encourage new users, that is what we're up against...

So our docs, our filenames, version numbers - all need to be simple and easy to get, at a glance.