Yet another re-write of "init".

[gyrog]

I had meant to wait until I had a script that would boot all the way in pupmode=5 before announcing this. But I want to make it clear that this is a lot more than just an "OverlayFS" support project.

It started as a re-write to cleaup the code so that it's more obvious and easier to work with. In hindsight, I think of the "init" that's in testing branch is the first draft, this is the second draft. You will see what I mean by that when I have a booting to pupmode=5 script, and make it available.

But it's also about trying a number of different things, one of these is to see what can be achieved using "Overlay" instead of aufs.

The initial release will be a minimum to boot pupmode=5, i.e. setup the stack with the sfs's and "switch_root" into that. Some things that are different already:

1. There is no searching at all.
2. Uses "blkid" to get the information about paritions.
3. The old "pfix=fsckp" is the default, the pfix parameter is now "pfix=nofsckp".
4. Has a revamped BOOT_SPECS processing that can be disabled with "pfix=nobs". While it normally overides real boot parameters, this can be changed so it does not.
5. Boot using the DISTRO_SPECS file in the install directory instead of the "initrd.gz". This also triggers using a BOOT_SPECS file in the install directory. This is the first step in trying to produce an "initrd.gz" that can be used to boot "any?" puppy.
6. A facility for CD/DVD booting to specify a BOOT_SPECS file using pupomode=77 technology. (not tested)
7. Introduces a new boot parameter "savemode=", to specify the pupmode and if there should be automatic saving in rc.shutdown for odd pupmodes. (only initial parameter processing done yet.)
8. A minor thing, but a number of binaries have been omitted from my test "initrd.gz".

My plans are to then introduce a save layer technology based on saving to an sfs file, running in pupmode=5 and maybe a modified pupmode=13. And then do it again using "Overlay".

[gyrog]

Well, the first draft is done, it pretty much does what I said it would do. The init script can be downloaded from http://www.fishprogs.software/puppy/initrd/init. The initrd.gz can be downloaded from http://www.fishprogs.software/puppy/initrd/initrd.gz. It boots to a desktop, but it only supports pupmode=5, there is no point in doing a save on first shutdown.

I've provided an initrd.gz because it can boot most any recent woof-ce puppy, both 32 and 64 bit. It doesn't contain a DISTRO_SPECS, and is capable of reading the DISTRO_SPECS file from the frugal install directory which must be specified as a boot parameter, e.g. "pdrv=Work:/puppy/slacko/". So you can copy the DISTRO_SPECS file to the frugal install directory, and replace the existing initrd.gz with the downloaded one, and boot.

The code is a bit different to what is normally seen in puppy, it tries to use functions in a more C like fashion. It tries to avoid using lots of global variabes that can be set anywhere else. The principle is that a function returns a single variable and it's return code. The return variable is called "function_name_RET", if the return code is 0 the return variable contains good data, else the return variable contains an error message. Although for some functions the return variable either contains good data or nothing. Within a function local variables are declared as "local". The idea is that when you are trying to workout how a block of code works, you can easily identify which data comes from elsewhere, and which is local.

I have not pushed it to rationalise yet, because I might just completely re-arrange it at any time. But I will continue to upload the current working version to my web site.

[wdlkmpx]

I only grabbed the init script.

It did not source the DISTRO_SPECS in / (this doesn't look right) So I had to specify pdrv=sda1

Then apparently it got the DISTRO_SPECS from somewhere but didn't find main dpup sfs, it printed the whole correct path without the leading '/'

[wdlkmpx]

gyro you can use https://github.com/puppylinux-woof-CE/initrd_progs . i just updated it

use this script: https://github.com/puppylinux-woof-CE/woof-CE/blob/testing/ztools/gitrepo

./gitrepo https://github.com/puppylinux-woof-CE/initrd_progs

select option 2, enter your github info. and in 2 seconds you're ready to push changes.

that way everybody can see how it evolves and test the script. you really should be using the latest initrd. just run ./build.sh -prebuilt -auto and in 10-30 seconds you have the latest initrd stuff ready to be used in your current install.

[gyrog]

@wdlkmpx, thanks for testing. I added the DISTRO_SPECS file into my initrd.gz, and booted slacko 6.9.9.6, no problems. However I did find another bug whereby it didn't honour "pfix=nocopy", now fixed. So I've updated new versions of init and initrd.gz. No, I will use the the initrd.gz I've got, It's small, minimiist, and it won't change. My only interest at this point is, does the "init" code do what it's supposed to do?

[wdlkmpx]

I still have the same issues or maybe i'm not providing the correct params. I'll try the initrd.gz later.

[gyrog]

My current version of this init users overlays instead of aufs. See this this topic in this forum topic.

[wdlkmpx]

I don't have a "huge" kernel with overlay.ko. So i used a debian kernel in 3builddistro, and inserted a few lines here and there. It booted to desktop.

[peabee]

The 4.9.13 huge kernel from xenialpup-7.0.8.1 comes with overlayfs turned on - see: http://www.murga-linux.com/puppy/viewtopic.php?p=957140#957140

[gyrog]

Yep, so far overlays have been working fine in xenialpup-7.0.8.1. My biggest waste of time was wondering why things did not work in "init" that worked fine in a running xenialpup, until it dawned on me that it's a module, I needed to insmod it in "init" before I could use it, duh!! The significant problem with overlays is it's limitation, I have yet to find any evidence that an existing overlay stack can be modified.

[gyrog]

Thanks to peebee, the 4.11.4 kernel in LxPupSc-17.06.21T, also supports overlayfs, see: http://www.murga-linux.com/puppy/viewtopic.php?t=101527&start=722.

Also a new version of "init" and "initrd.gz" has been uploaded see: http://www.murga-linux.com/puppy/viewtopic.php?p=957676#957676 See second post above for download url's.

[gyrog]

Thanks 01micko, the 4.4.70 kernel in slacko 6.9.9.9, also supports overlayfs.

[wdlkmpx]

I'll continue testing the new init in july (most likely), but i'll sure check the forum thread to see the new updates and notes. at the moment i'm dealing with stuff i downloaded years ago and need a suitable music player for that purpose (e.g: https://github.com/Alexey-Yakovenko/deadbeef/commit/1762628473b4ee517ff8e5db884288af514f6121) ..

[gyrog]

To compile a kernel that supports overlayfs, the following lines should be included in the kernel config: CONFIG_OVERLAY_FS=m CONFIG_TMPFS_XATTR=y

[wdlkmpx]

There's a cool feature that's worth implementing in the current and the new init

The ability to test an iso without a cd drive, virtualbox, qemu, etc is tempting.. I have an arch ISO and a grub4dos menu that make this happen:

title Arch Linux [2015.11.01] i686\nInstall Arch Linux
  set isofile=/ISO/archlinux-2015.11.01-dual.iso
  set isolabel=ARCH_201511
  find --set-root %isofile%
  uuid () > nul
  set UUID=%?%
  map %isofile% (0xff) || map --mem %isofile% (0xff)
  map --hook
  root (0xff)
  kernel /arch/boot/i686/vmlinuz archisolabel=%isolabel% img_dev=/dev/disk/by-uuid/%UUID% img_loop=%isofile% earlymodules=loop archisodevice=/dev/loop0
  initrd /arch/boot/i686/archiso.img

We can see here that the init script accepts some unusual parameters: archisodevice="" img_loop=%isofile%,etc

grub4dos: 1) finds the iso and sets the root partition and directory 2) gets the UUID of the current partition 3) maps/opens/mounts/loads the iso file 4) sets (isofile)/ as root dir 5) loads initrd and kernel

When you boot a linux system that way, there is no CD device. So you have to pass all the required info as params... so that the init script can find, mount the iso and load stuff... or so I guess.. I'm not sure.

Although some other ISOS just work and don't require those quirky params... hmmm..

But puppy certainly doesn't work, that's why the above grub4dos menu serves as a guide..

[gyrog]

Puppy has ISObooter

[wdlkmpx]

Here are some thoughts after reading the thread about the new init.

Puppy users are used to run many different puppies, with a different desktop, different apps, kernel, xorg version, woof version, compat-distro version, etc.

All this nonsense instead of unifying puppy with alternative kernels, xorg versions and desktop environments installable "out of the box"... -*- my approach for myself and the "alternative" code i introduced directly in woof.

You can't take such amount of inconsistencies seriously, but you can view puppy as your personal rescue distro, and it starts to make sense.

The init script is puppy itself, therefore it must be complex to appease the masses with bread and circuses. Many other parts of the puppy stuff can be safely simplified, and they already are.

Yet still the init script itself can look clean by basically moving most stuff to individual scripts in /bin and /sbin. Some modularity that with some basic knowledge can be simplified by removing components and the lines calling them.

Over time i realised that a big script can be easily understood if it's well designed, so the number of lines is not as important as the readability, the less cryptic the better unless it's to considerably improve performance.

-*- Visions. Conflicts. Philosophies. It could have been. Here is the main problem and the deal-breaker, but the best way to troubleshoot this to "sell" the new features (or just to avoid losing them) is to add a parallel implementation that can be easily triggered in the standard build... that's what I do. Over time your changes can become the default behavior, or even the only behavior possible, etc... but hard to know what to do when there is a long history of puppy tradition and madness. I added the grub4dos iso with alternative kernel support in the woof process, no logos, no customizations, support for pet/deb/tgz/txz kernels, etc.

It's the highway, but the puppy philosophy of the "one-man show" demands it. A sense of masochism must be present, and it must be stronger than the night.

Tradition vs Common sense.. must have both for different types of people.

Windows FS support in the initrd. I use a NTFS-formatted usb flash drive with all my puppy live stuff, linux and windows isos/installers.

With the latest release of ntfs-3g and dosfstools running puppy in such partitions is waaaay more reliable now. Many old PCs only boot with a vfat partition, specially usb flash drives. With a 2mb initrd.gz.. it's not that big. It should be initrd.xz, the build scripts and edit-initramfs can handle both.

Probably the best approach for new experimental stuff (if you dont want to use the 'rationalise' branch) is to create a new branch derived from testing:

git checkout testing git pull https://github.com/puppylinux-woof-CE/woof-CE testing git checkout -b mybranch

And continue with the work ... (do changes) ... git push https://github.com/puppylinux-woof-CE/woof-CE mybranch

To sync both branches keeping the changes in 'mybranch' on top:

git checkout testing git pull https://github.com/puppylinux-woof-CE/woof-CE testing git checkout mybranch git rebase -i testing ... (steps) ... git push -f https://github.com/puppylinux-woof-CE/woof-CE mybranch

This might or might not require manual edits to fix some conflicts.. depends on what you edit and what other people are editing in testing.

Such a branch is unstable by definition due the rebase process. But it will always look clean and up-to-date. Such a branch should not be touched nor cloned by people who don't know how to use git. You can edit commit messages, squash commits into one, everything is possible. In the end a pull request or a merge operation is enough to make your changes available in the standard builds. It's no secret that building a woofce puppy is easy, you can build your puppy from a specific branch for everyone to test your changes "out of the box". Or you can ask someone to do it and provide a download link.. it's really a matter of comunication...

Other than that, there's plenty of time to think about how to improve the scripts. So whenever you think something is ready for testing, rationalise, etc... just do it. Taking -*- into consideration.

[wdlkmpx]

Gyro. How about integrating your new init and scripts through a kernel param pfix=altinit or something. I suggest this because it will see limited testing. I usually only edit filez rather rewriting them.. in the end the result can be a complete rewrite, but i usually try to keep the same behavior to avoid the backchat from the p***y community.. but at this point i don't think it really matterz.

I suggest doing it in testing so the chances of getting the new init stabilized are higher. I honestly dont have time anymore but i can help i guess. I know this requires communication and thought but things happen here because someone decides to do or change something, not because of agreement.

About the changes to happen, they can happen as many timez as needed, i believe the git repo should be archived, renamed and restarted because there is too much junk and big files in the git history anyway. Committing from android..

I said it before, the new init can enable the use of untouched distro kernels as a new method. I've seen it work. But i probably Won't test it againt until next year.

[gyrog]

Unfortunately, we are not all on the same page. I see the "initrd.gz" as a restricted environment that should be exited as soon as possible. So the "init" script should be a bare minimum, and "initrd.gz" should support the "init" script and nothing more.

My plan for the next version in this series is to do a skeleton "init" that contains just enough to be usefull as a "production" "init".

"DISTRO_SPECS" will have to be in the "initrd.gz". "BOOT_SPECS" support will be provided for writeable devices like HD, SSD, usb sticks etc..., and multisession DVD, and isoBooter.

It will support "psfspart=", "psfsdir=", "psavepart=" and "psavedir=" boot parameters. There will be no support for "pdrv=", "fdrv=", "zdrv=", "adrv=" or "ydrv=" boot parameters. There will be support for "pfix=nosave" and "pfix=nobs" boot parameters.

There will be a utility to manage 3 sfs lists, "SFS_ABOVE", "SFS_MAIN" "SFS_BELOW". The sfs files can reside anywhere, on internal or usb partitions. The "init" script will load these into a "pup_sfs" overlay stack in the above order. But the lists will be mounted in the order, "SFS_MAIN", "SFS_ABOVE", "SFS_BELOW". It is assumed that the first entry in "SFS_MAIN" is the puppy...sfs file. The boot will be aborted if it fails to mount. If "overlayFs" support is a kernel module, it is assumed to reside in the last entry in "SFS_MAIN". The default lists are defined as: P_SFS_ABOVE=":$DISTRO_ADRVSFS,:$DISTRO_YDRVSFS" P_SFS_MAIN=":$DISTRO_PUPPYSFS,:$DISTRO_FDRVSFS,:$DISTRO_ZDRVSFS" P_SFS_BELOW=''

Usage: Either "psfspart="(or "pupsfs=" or "pdev1=") or "pmedia=cd" must be defined as boot parameters. Usually, neither "psavepart=" nor "psavedir" would be specified as a boot parameter. First boot will always be "pupmode=5". It will continue to boot in "pupmode=5" until "P_SAVE_PART" is defined. It will then run in "pupmode=12" using a savefolder based on "P_SAVE_PART" and "P_SAVE_DIR". There will be a utility to choose "P_SAVE_PART" and "P_SAVE_DIR". "P_SAVE_PART" can be a normal Linux partition or a Luks partition.

Possible addon version: Automatic persistence, including multisession DVD. This will be done by making a tar of the "pup_rw" in tmpfs on shutdown, and restoring it in "init". (Similar to the current version of this project.)

Concepts:

1. Coding conventions to make really global variables more obvious.
2. No searching. (Almost, "pmedia=cd" uses the first 'iso9660' partition as P_SFS_PART.)
3. Use overlayfs.
4. Use utilities in the running system to manage many boot options. (BOOT_SPECS)
5. Place savefolder on any normal Linux or Lucks partition in the computer, including a second usb device.
6. Any number of sfs files can be added to any sfs list, and these sfs files can reside anywhere.

Issues: "wait4usb" in woof-ce only waits for the first usb storage device to be available. If 2 usb storage devices are attached, the second device could be ignored. This project uses a "wait4usb" that includes an extra 1 second wait after the first device is found, to provide a chance for any second device to be detected. It would be nice to have a "wait4usb" that always waits for all usb storage devices.

Long term plan: Finish this as an alternate "initrd.gz" + "ydrv_overlay.sfs" package, (so I can start using it as my "production" initrd). Then hopefully have a discussion of it's woof-ce possibilities;

1. Port some of it's features to the current "init".
2. Write an aufs version of it, place it on woof-ce, and port extra stuff from current "init" to it, eventually replacing current "init".
3. Include it in woof-ce as an alternate overlayfs initrd.

I intend to write a document outlining the differences, including "ease of porting" comments.

[wdlkmpx]

I'm ok with that.

I dont mind specifying params for specific situations, but i guess most users will not be aware of the changes and adapt. In the official Project threads, people often seek help regarding the changes in the init, or so it seemed to be.

Thats why i also wouldnt mind having a init with addons controlled by variables, so one can decide what stuff is to be executed.. although kernel params can do the same.

Well, we all have different views and as absurd as it seems, people who are forced to like rox against their will, often find themselves trying to git rid of better file managers (all the others), when someone fought for their miserable freedom..

[gyrog]

Sorry if the following covers some stuff I've mentioned above, but it's been a while, and I've just spent some time writing it in geany, so I'm going to post it anyway.

---

I've just released overlay_init-0.1 in the forum.

Some interesting differences between it's internals and the 'init' in 'testing': (I'm not sure this is a complete list.)

1. While they both implement the same basic algorithm, this 'init' has a much cleaner set of functions.

2. This 'init' code attempts to reduce the problem of the sudden use of a variable and you have no clue as to where in the code it might have been last set. An aid to this is the use of 'local' variables, and identifying the use of variables that are actually used globally. It also introduces a coding convention I will refer to as "string functions". With "string functions" the numeric return code is always ignored. They return 1 or more strings with the form, string_func_name_RET, or string_func_name_ERR, or string_func_name_PART. Their success is indicated by non empty main return string e.g. [ "$string_func_name_RET" ] || error-exit "$string_func_name_ERR". These return strings are only ever set in the function with the same name, and they are always preset to '' in the function. The idea is that when you see one of these variables used, you know which code set it, and that it's contents relate to the most recent invocation of the function. That invocation of the function should be in code not far above the use of the variable.

3. This is a 'probepart' free 'init', it depends solely on 'blkid' for information about partitions.

4. The 'initrd.gz' is minimilised to contain only the binaries necessary to run 'init'

5. The PREVUNIONRECORD and LASTUNIONRECORD variables in BOOTCONFIG, do not contain a list of sfs names. They are a list of last modification times for the sfs's as 'seconds since the epoch'. This is so 'rc.update' can detect an sfs that has been replaced with one with the same name. The last modification time for "/lib/modules/$KERNELVER/modules.order" is appended so 'rc.update' can detect if the current change involves a kernel change.

6. The 'pup_ro1', 'pup_ro2' convention has been obselete since 'adrv' and 'ydrv' was introduced, it is abandoned. Sfs files are mounted at /pup_sfs, where corresponds to number of the loop device used. So the first sfs mounted, the puppy...sfs file, gets /dev/loop0 and is mounted at pup_sfs0. Any following sfs files get an in order of their mounting. Note: This does not corrrespond to their order in the stack.

7. All the sfs files are loaded into a read-only overlay mounted at /pup_sfs. This is to provide a mechanism to easily see if a file resides in any sfs in the stack.

8. The "/mnt/dev_save" convention is abandoned. All partitions are mounted at /mnt/ e.g. /dev/sdb2 gets mounted at /mnt/sdb2. An exception is Luks partitions, /dev/mapper/sdc3 gets mounted at /mnt/mapper+sdc3. Note: 'rc.sysinit' still manages to get '/mnt/home' correct.

9. The text file BOOT_SPECS is used as a communication mechanism between utilities in the running system and 'init'. BOOT_SPECS usually resides in the install directory, but can be moved elsewhere by setting the default 'save' location with 'psavepart' and 'psavedir' as actual boot parameters.

10. It introduces a PBOOTSPECS variable to PUPSTATE, to ensure that 'init' and the utilities are using the same copy of BOOT_SPECS.

11. It introduces a new pupmode=21. It uses a unique pupmode number so that code unique to it can be added to Puppy utilities without clashing.

12. '/tmp/bootinit.log' contains a running commentary on what 'init' is doing. Messages that appear on the console, also appear here.

13. In the running system, '/initrd/' contains a copy of the significant files that were used by 'init.

14. There is no searching in this 'init'. The only time it comes close is when booting 'pmedia=cd', e.g. when booting from a CD/DVD. If no SFS_PART is defined, it will use the last iso9660 partition it knows about. The idea of using the last, is so that an actual CD in 'sr0' takes precedence over an 'sdc4' partition from isoBooter.

Where to from here?

A first issue is to patch some puppy utilities so that they work with the current 'init' in 'testing' and this new 'init'. Patched versions of the following files are contained in 'overlay_mods.sfs' The obvious ones are: rc.shutdown rc.update initmodules The not so obvious ones (that I have found so far): petget/installmodes.sh pup_event/frontend_funcs pup_event/frontend_timeout

The next issue is, what extra facilities do I intend to add? I intend to extend pupmode=21 so that it also supports multi-session DVD's, a bit like pupmode=77. I might investigate a possible hybrid version that uses an aufs stack but the only read-only branch is a read-only sfs overlay.

The last issue is, should we, port some of it's stuff to the current 'init', put it somewhere in woof-ce so everyone has access to modify it and port "needed" stuff to it, or put it in woof-ce, re-write it to use aufs instead of overlayfs, then port all the "missing" stuff to it, or leave it as a gyro specific alternate 'init'. Or, could it be added to woof-ce as an alternate 'init', selected at build time, i.e. you use woof-ce to build an 'artful' pup with aufs, or an 'artful' pup with overlayfs.

[wdlkmpx]

I edited pup_event stuff, hmmm..

I like the idea of porting stuff, that's how a rewrite starts, i try to avoid completely rewriting a script.. the obvious reason is to avoid a lot of debugging and a burden no one wants to carry. Specially in the most unhealthy distro ever.

I'm a Windows user most of the time. p is my rescue distro and for old unusable hardware, but not older than a Pentium 4.

The scripts i edit.. some of them completely acquire my coding style, which is closer to C, so it can classify as a rewrite. Although it might seem circular, all my edits can lead to a huge redesign when common sense triumphs.. probably never.

I already talked about addons and stuff, how the main init can trigger other inits by checking the available FS's and configs (no aufs? -> overlayfs (default distro kernels)), how can boot params force specific stuff. Everything is possible, ideas flood my mind. What i don't have is time. I'm losing money and i'd rather learn how to carry bricks and do something that actually benefits me

You can use testing and add your stuff as an alternate init that triggers alternate scripts or an alternate behavior within the usual scripts, that way i can directly help you with any doubt or request you may have. Just like my alternate grub4dos iso, i wanted to wipe the default non-uefi syslinux iso, but then i thought to myself.. i'll be bullied.

My choice is always the 'testing' branch for a reason, my 'w' branch is stalled and those changes might get lost in time.. and that means a lot of wasted time.

[gyrog]

Unfortunately for me, this is the 'init' I wish I had written for 'rationalise'. But I do completely understand the huge work load of testing required to introduce a new init. We did it once, do we really want to do it again? Probably not.

Puppy is my workhorse OS, I use it for most things, including writing Windows applications. So, I'm looking for stability, flexability, and still working after there are no aufs patches for new kernels.

When this 'init' is working as the boot mechanism for my workhorse Puppy, I will look at integrating it into woof-ce. But first I want to look at patching some Puppy utilities so that they are compatible with both current and new. This usually involves reworking hard coded references to things as they are now. e.g. replacing references to "/initrd/pup_ro2" with "/initrd$PUP_LAYER"

[gyrog]

Strangely perhaps, I am not familiar with the woof-ce build process at all.

So, is it feasible to provide a build option in woof-ce that is either "aufs" or "overlayfs" ? Noting that the "overlay" option might require, not only a different "init" script, but also possibly a reduced set of binaries in the "iniitrd.gz". And also possiblly different versions of some Puppy utilities. Although hopefully this will be minimal list. And maybe the ommitting of some utiltites, e.g. sfs_load, snapmergepuppy.

[wdlkmpx]

Building is easy, anyone can do that. It used to be slow as hell, but now it's fast even though the whole design is more suitable for an independent distro like Puppy 4, Wary or Racy. You just need about 6-10gb of free space and that's enough.

https://github.com/puppylinux-woof-CE/woof-CE/wiki/Git---users

When building just press enter or click on ok on any prompt. It can be automated a bit more.

To keep your wce install up to date, just build the puppy of your choice, then replace the sfs's, initrd.gz and vmlinuz, make sure there's no conflicting stuff in the pupsave (or remove it) and boot again. That's what i do.

Always read the commit titles, and read the commit diffs if possible.. that's what i do.

I think always assuming aufs by default is ok.

A new build option for overlayfs is easy to add. In _00build.conf: UNIONFS=overlay

This will trigger stuff, such as adding DISTRO_UNIONFS=overlay to DISTRO_SPECS and all the stuff you want it to do.

Just add DISTRO_UNIONFS=overlay to your test initrd.gz and make sure it boots to desktop with alternate stuff (the init can configure rootfs before switching root, this can also in 3builddistro), that's all, you can add code directly to testing that way.

[wdlkmpx]

gyro You might want to add your custom scripts to rootfs, if there's no conflict then you can place them there until you enable code to use them, so that your code is already in newer builds.

init_full_install is a good example of how to integrate a new init that is triggered by the main init.

DISTRO_UNIONFS=overlay in DISTRO_SPECS should be a flag to be read only by the initrd init script.

The init script should write UNIONFS=overlay to PUPSTATE if overlayfs is actually being used.

Scripts in rootfs should only rely on /etc/rc.d/PUPSTATE to get the UNIONFS.

And scripts could refuse to run or call to specific apps if UNIONFS=overlay or UNIONFS=something_else

That is a simple mechanism to trigger the desired behavior.

On second thought, if you're not confident enough to do it in testing, rationalise might be the place where you can make some large or sensitive changes that might break the default behavior, then peabee can help spot broken stuff... having verified that change is not bad, then a merge operation happens in testing.

In fact i think tomorrow i'll be pushing my changes for the pup_event stuff to rationalise, hoping some more testing happens (i'm going into maintenance mode), these changes are not alternative stuff, this is a sensitive replacement, but with the same functionality, it's actually a simplification, it's a wip, but i'll just leave as it is now.

[woodenshoe-wi]

I don't know if this would help with working on changes without actually adding them to woof-CE, 66f98da

But it might not be what you are talking about at all.

[woodenshoe-wi]

Here is an example of what I was thinking. merge2out-gui.tar.xz and EXTRA_MERGE-overlay_init-0.1.tar.xz

merge2out-gui.tar.xz contains a modified merge2out and woof-code/_00func plus a new merge2out-gui and .gitignore - these would have to be added to woof-CE and would enable extra directories to be included during the merge2out process.

EXTRA_MERGE-overlay_init-0.1.tar.xz contains two subdirectories, initrd-progs and rootfs-skeleton. The initrd-progs directory will be merged with the one from woof-CE, replacing the init in 0initrd with the one I took out of your overlay_init-0.1.tar. The rootfs-skeleton directory contains everything from overlay_mods.sfs

When running merge2out-gui select overlay_init-0.1 as the EXTRA_MERGE directory and continue the build as normal. When 3builddistro-Z finishes, the initrd.gz will have the new init and everything from overlay_mods.sfs will already be in the main sfs.

There will be more stuff in the initrd.gz that 3builddistro-Z creates than initrd.overlay.gz, because this approach overwrites files in 0initrd but cannot delete them. If this is a problem I can try rewriting the function in _00func to completely replace the 0initrd directory instead of overwriting individual files.

I did a test build with this setup, and after manually creating the BOOT_SPECS file I was able to get the session to save and also was able to load the devx.

[gyrog]

@wdlkmpx I intend tp start pushing patched utilities soon, but I haven't yet checked that the patched code still works in a normal Puppy. Any code that is peculiar to "overlay" will not interfere with "aufs" functionality. This is one of the reasons I have introduced the "archive" save mechanism with a unique pupmode.

I'm hoping that the configuration: "_00build.conf: UNIONFS=overlay", can result in the production of an "initrd.gz" that contains just the "overlay" init and required support binaries.

@woodenshoe-wi What you are suggesting looks interesting. I need a little time to digest it. While being able to reject some files in "0initrd" would be nice, being able to replace the default files with "overlay" specific files would be a good start. Strangely, a read-only overlay with the "initrd.overlay" on top and "0initrd" underneath would work quite nicely. Just a thought, if we have a configuration option "UNIONFS=overlay", couldn't we just have some code that deletes some binaries from "initrd.overlay" if this configuration is "overlay", if it's really necessary.

[gyrog]

@woodenshoe-wi I'm not sure if you expect the "EXTRA_MERGE" directory to be purely local, or located in woof-ce. I want this "init" and it's supporting files to be located in woof-ce. So I'm suggesting a possible different directory structure with "extra-merge" being a top level diretory in woof-ce to hold "maintained" EXTRA_MERGE directories: extra-merge/initrd-overlay/initrd-progs/0initrd/init extra-merge/initrd-overlay/woof-code/rootfs-skeleton/etc/rc.d/rc.shutdown extra-merge/initrd-overlay/woof-code/rootfs-skeleton/etc/rc.d/rc.update

[wdlkmpx]

Hmm ... i think EXTRA_MERGE might benefit someone doing stuff outside woofce and test building isos replacing files, so regardless whether it's used for overlay stuff.. it still might be useful to someone..

Well, gyro, stuff has changed a bit in rootfs, and i'm about to push some changes to finish what i started, after that there will be peace.

In the past there was code for the old unionfs alongside aufs, i think this might be the best choice.. adding ifs, but if you want to delete stuff, then so be it.

Hmmm .. You can also add your files with .overlay suffix, then it will make sense that UNIONFS=overlay in 00build.conf makes 3builddistro search for .overlay files, rename them deleting the suffix and replacing the original file.

You can also paste here what actions you expect to happen, i.e: commands to be added to 3builddistro..

[woodenshoe-wi]

The EXTRA_MERGE stuff was meant for doing stuff outside of woof-CE like wdlkmpx said.

It was an idea I had some time back, inspired by some comments from peabee if I remember correctly, that I decided to try to implement now.

If you want the alternate init inside of woof-CE I think it would be simpler to modify initrd-progs/build.sh. It could source ../_00build.conf and if you want a different directory structure than the original init there could be a initrd-progs/0initrd.overlayfs directory that contains all the arch independent parts of the initrd. Just remember that if you want to still support ARM chips there should not be any compiled binaries in 0initrd.overlayfs, build.sh has to copy in binaries of the right arch.

[wdlkmpx]

Ok today is the day. rationalise and testing are ready, how should we proceed to start this

[wdlkmpx]

I see files in the forum thread, not sure where to place them or how to trigger them, but i'll add dummy stuff .. #UNIONFS=overlay in _00build.conf

If those files are replacements for the the current scripts in a overlayfs based system, i obviously won't edit them unless i start to use overlay.

And for that to happen, I'll have to use a compat distro kernel properly configured in 3builddistro.. yes, exactly that, overlay is so primitive it's only useful when aufs is not present by default.

They say aufs was rejected because of its messy code, but in this case it was better to clean it up instead of merging overlay into the kernel

[wdlkmpx]

I added the dummy stuff, files can be added anywhere (in testing or rationalise... up to gyro), and i can (mechanically) follow instructions to make things work or gyro or woodenshoe-wi may apply them themselves. I'll test the new stuff a few months later

[gyrog]

So, the EXTRA_MERGE stuff is meant to be used creating an iso using stuff outsied woof-ce, so I'll leave that aside.

@wdlkmpx I still need to come to grips with the way woof works before I can make any sensible use of the dummy stuff you have done, but I intend to.

The changes to current Puppy scripts can be incoprorated into the normal code, and activated by PUNIONFS='overlay' or a unique PUPMODE. I can start doing this in 'rationalise' ASAP.

There can be a common set of initrd binaries in woof-ce that will work for both, I curently remove some unused stuff to simply save space.

So, requirements for building an "overlay" iso would come down to:

1. Replacing the default "init" script with the overaly "init".
2. Including the overlay support utility scripts that do not exist in the default iso. While the CLI scripts could be simply included in woof-ce, some GUI scripts should not appear in a non-overlay Puppy menu.

[wdlkmpx]

If you want i can get this started my way, but it has to be today. Then you can suggest/apply changes

Introducing Githubissues.

Githubissues is a development platform for aggregating issues.