blue-build / website

Website that hosts BlueBuild documentation.
https://blue-build.org/
Apache License 2.0
9 stars 7 forks source link

docs: Additionnal information on how to do some things (systemd|ansible|fetch file from web) #36

Closed johnr14 closed 7 months ago

johnr14 commented 7 months ago

Ok, I got my ublue to build and running. The ISO builds and installation works.

I am finding it hard to find information on how to configure the system after. I am thinking like Nix or Ansible fine tuning.

Use case:

I will run ublue on my PCs as well as my relatives that I am slowly moving away from Windows. Ublue will allow me to not manage those computers personally. While they can install their own flatpak, it's going to be easier to manage them from my recipe.yml.

So now, the information missing in the documentation is mainly :

systemd : enable/disable a service by default

Example : sshd seems to be enabled by default (on first installation). How could I disable it ? From the recipe.yml or with a script/sshd.sh ? From what I know, a link like this should do the work, but how to bundle it ? ln -s /etc/systemd/system/post_deploy.service etc/systemd/system/multi-user.target.wants/

ansible : run playbook to configure the system

Changes to /etc on the ublue ghcr.io will not overwrite local /etc, so ansible would be needed for that, right ? How to get recipe.yml to fetch latest ansible playbook from git and save it to /usr/var/local/ansible_playbook ? Example : run a playbook on first boot to configure system specific configs. Fetching from github and running the playbook_firstboot.yml on new installation. Example2 : run a playbook_rebased.yml after rpm-ostree rebased to a new version, update github copy and run playbook. This is so I can propagate changes to all computers without having to punch a hole in firewall or knowing where the laptop is at. Things like preventing sleep of laptop when on power.

appimage : fetch latest appimage

While flatpak are great, there are a few app that ships a .appimage file. How could I have it fetched a fresh version and saved in /usr/local/bin for updates with every rpm-ostree upgrade ? Example: dvdisater

multiple images from single codebase

While there is some information, a complete example would be better with individual file view like this.

Great work ! Thanks !

gerblesh commented 7 months ago

systemd : enable/disable a service by default

there is a specific systemd module that handles this: https://blue-build.org/reference/modules/systemd/

ansible : run playbook to configure the system

while we don't have specific support for this, you can use a script module, or make your own https://blue-build.org/reference/modules/script/ https://blue-build.org/reference/module/

appimage : fetch latest appimage

same here, just using the script module would probably be sufficient

multiple images from single codebase

not sure if the docs explained it, but you can define modules in different files and use from-file: path/to/yaml.yml to source them.

xynydev commented 7 months ago

I'm gonna answer in the same way as gerblesh did, repeating some points and adding some of my own.

systemd

Looking at the list of modules on the website is a great way to figure out what things are supported easily OOTB. The search on the website also works pretty well. There's a systemd module for doing exactly what you describe: https://blue-build.org/reference/modules/systemd/

ansible

While first-class support for ansible isn't planned, the idea of having a module for running scripts at first boot has floated around.

That being said, changes to the /usr/etc/ in the image do progagate to image users' /etc/ if they haven't edited that specific file locally themselves. I can't of course be certain about it, but I find it unlike that your relative will go and edit a file in /etc/.

You can't save an ansible playbook to /usr/var/local/ (first because that doesn't exist, you probably meant /var/ or /usr/local/), but also because that is the system administrators territory to be changed locally. You could make a script to do a git clone for the playbook into /usr/share/ansible-playbook/ or some other custom folder like that.

I'm unsure what you ask to be done, documentation-wise, about this.

Appimage

I'm unfamiliar with what the runtime requirements are to run Appimages, or how Appimages can be used of atomic Fedora.

They can be fetched into /usr/bin/ (not /usr/local/bin/, again, because that is local sysadmin territory and not included in custom images) using a script. The exact mechanics of how to fetch them vary wildly depending on the platform used to distribute them. GitHub releases unfortunately makes it kind of hard to always fetch the latest version, so if you don't want to figure that out you can just download a specific numbered version, or request that a module be made for the usecase and hope that gets implemented soon.

I'm unsure what you ask to be done, documentation-wise, about this. This page already covers some of the directory-related things I talk about: https://blue-build.org/learn/mindset/

Multiple images

I believe the docs already include the necessary amount of information to get started with this, but more can be added of course

I believe you're asking for examples of contents of specific files to be added here? That wouldn't be a problem, should be a pretty quick addition.

johnr14 commented 7 months ago

Well, I did miss the systemd module on the website ! Thanks for the quick replies.

I've split this in a few post for redability.

yafti

Could yafti have it's name written in long in the doc ? "Yet Another First Time Installer". From what a quick read got me, it looks like a program (gui?) to suggest installing other things on first boot. More could be added in the documentation in regards to what it does and what it can do. Doesn't seem to be what I am looking for.

Missing modules scripts_firstboot and scripts_postdeploy ?

+1 for module scripts_firstboot: and scripts_postdeploy: ! It could be used to run ansible playbooks or any other scripts only when a) new install b) ostree deployment. This should be added if ansible can't be a first class citizen any time soon.

Also, It seems firstboot: module exist and is used with yafti ??? It's undocumented ? Can this work ?

firstboot:
   modules:
      - type: scripts
        script:
          - firstboot.sh

Thanks

xynydev commented 7 months ago

Also, It seems firstboot: module exist and is used with yafti ??? It's undocumented ?

The yafti module has a firstboot script to launch yafti, yes, but a firstboot module would use it's own. So no, your solution below wouldn't work.

Also, the yafti module currently links to the GitHub page, which has more information, but I do agree that the module page could give a brief explanation of what yafti is.

xynydev commented 7 months ago

Here's a checklist of the actionable things in this issue:

After those are completed, the issue could be closed, correct?

There's already an issue tracking the implementation of a potential boot-script module: https://github.com/ublue-os/bling/issues/119

johnr14 commented 7 months ago

That's 2, missing about running something on first run after rebase.

I am working on a systemd .service and .timer to add as example. The specific is for a service that will run only on a reboot after a rebase but I can't get it enabled in the recipe.yml, build gives this :

Failed to enable unit, unit rpm-ostreed-deployed.timer does not exist.

The recipe.yml part :

 - type: systemd
    system:
      enabled:
        - sshd.service
        - rpm-ostreed-deployed.timer
      disabled:
        - rpm-ostree-countme.timer # RedHat telemetry disabling  
        - rpm-ostree-countme.service
      masked:
        - rpm-ostree-countme.service
        - rpm-ostree-countme.timer

Files : config/files/usr/etc/systemd/system/rpm-ostreed-deployed.service

[Unit]    
Description=Run once on reboot to check if a rpm-ostree was deployed   
Wants=rpm-ostreed-deployed.timer  

[Service]
Type=simple
ExecStartPre=/usr/bin/env bash -c 'journalctl --boot=-1 -u ostree-finalized-staged.service | grep "Finalizing staged deployment"'
#return 0 if there was a deployment and 1 if there was no deployment
ExecStart=/usr/bin/env bash -c 'echo "There was a deployment and this is the firstboot!" > /tmp/firstboot_post_deployment.txt'

Edit: fix journalctl to include -u ostree-finalized-staged.service instead of an other grep that can give false results.

config/files/usr/etc/systemd/system/rpm-ostreed-deployed.timer

[Unit]
Description=Run on boot

[Timer]
OnBootSec=10
Unit=rpm-ostreed-deployed.service

[Install]
# To have it run only at boot
WantedBy=multi-user.target

It seems that files copied with :

modules:
  - type: files
    files:
      - usr: /usr

can't be enabled for a new service ?

xynydev commented 7 months ago

missing about running something on first run after rebase.

That would be a part of the boot-script module I mentioned, I think. I'm not sure if that hack to check if this is the first deployment is the best way, but thanks for doing some hacking on it anyway. Yafti uses a file it places in a local directory to check if it has been run previously. Something like that would work better for multi-user systems, where the first-time script should run for everyone. Ideally, everyone should write boot scripts that can run on every boot, which would accommodate both not being run only once and updates to the script itself being run again on a system.

It seems that files copied with : [...] can't be enabled for a new service ?

Hmm, they should be. The files should be in /usr/lib/systemd/, though. Maybe that should be documented on the systemd module page... And the files module notice should be expanded to note that no program reads from /usr/etc/ so if a configuration file is needed at build time it should be placed in another directory like /usr/lib/ or /etc/

johnr14 commented 7 months ago

Hmm, they should be. The files should be in /usr/lib/systemd/, though. Maybe that should be documented on the systemd module page... And the files module notice should be expanded to note that no program reads from /usr/etc/ so if a configuration file is needed at build time it should be placed in another directory like /usr/lib/ or /etc/

Ah ! that may be why, as they appear in /etc on the deployed vm but not available during build. Will try to usr/lib/systemd/system ... waiting for build and deploy... Ok, builds and I'm able to enable the .timer and it does work.

Thanks

Well, my hack could allow a service to run on every rebase. Like run git pull and ansible.

Could be an example that might be useful until a boot-script module is added. You are free to add it to the documentation as an interesting example :)

Bonus :

Firstboot check

Also, to check if it's a first boot after a fresh installation and not a rebase, it's possible to validate it with : journalctl --list-boots | wc -l | awk '$1>2{exit 1}'

Could use only one systemd file and use a script to identify if it's a rebase, fresh install, or just a reboot.

Using journalctl is the best way I can think of to check for rebase and first boot.

Also note on how to disable RedHat telemetry with countme, can be nice to let people know they have that enabled by default.

Thanks again.

Also, there was somewhere the info on how to build locally, but on the the howto it's empty ?

johnr14 commented 7 months ago

For the file module, it's not described how to set a file permission. For example, I added a /usr/bin/rpm-ostree-check file, but it doesn't have execute flag.

I made a little script set_files_permission.sh to just chmod a+x /usr/bin/rpm-ostree-check

Is this how it's supposed to be ? Seems tedious to do scripts for every command needed to be executed. But if it's how it should be, maybe just add it to the documentation.

I would have tried instinctively to do :

modules:
  - type: commands
    command:
      - chmod a+x /usr/bin/rpm-ostree-check
      - cd /usr/opt ; git clone some repository
xynydev commented 7 months ago

Also, there was somewhere the info on how to build locally, but on the the howto it's empty ?

Yess, the documentation is still unfinished.

how to set a file permission.

Not sure, I think git supports setting file perms and those propagate to the built image.

I would have tried instinctively to do...

This will be possible soon.

Also note on how to disable RedHat telemetry with countme, can be nice to let people know they have that enabled by default.

I think a lot of small things people might want to do to customize their image are ones I expect people to ask about and and answer on the forums: https://github.com/orgs/blue-build/discussions

Adding a lot of marginally useful content in the docs isn't the goal. This specific example could probably be tucked on the bottom of the Fedora/Ublue pages, but that would feel like an official endorsement for turning off telemetry.

xynydev commented 7 months ago

The docs part of this issue is now done, so closing.