danboid / ALEZ

Arch Linux Easy ZFS installer
GNU General Public License v3.0
145 stars 25 forks source link

ISO creation questions #42

Closed qdm12 closed 5 years ago

qdm12 commented 5 years ago

Hi there,

I am trying to build the custom ISO from my local source code. I noticed there is a Dockerfile. On the other hand, there is Boot into an up-to-date install of Arch (but not under LXD/LXC as the build script doesn't work in a container) in create-alez-iso.txt. So can we build the ISO in a container or not?

Another important point is that the Dockerfile builds the ISO by getting the source code from Github whereas it should really use local source code as one would git clone the whole repository anyway. Similarly, auto-updating ALEZ with the downloader is I think a bad idea and that is why you have releases (plus you already have them set up). What do you think? I can change all that in a first PR if you want, I'm quite familiar with Docker.

qdm12 commented 5 years ago

It seems this works nicely:

  1. Clone the repository

    git clone https://github.com/danboid/ALEZ.git
  2. Build the Docker image

    docker build -t alezbuild .
  3. Run the Docker container and bind mount the output directory to obtain the ISO file

    docker run -it --rm --privileged -v /yourpath:/opt/alez/iso/out alezbuild

    The ISO file is now in /yourpath

I will add it to the readme in a first PR if you don't mind.

johnramsden commented 5 years ago

ISO builds should work fine in containers, at least I haven't had any trouble in Docker containers, and that's how I set up the continuous integration builds. You'll probably have to ask @danboid about any further information he had on container builds, I've never tested LXC/LXD builds so I can't comment on why the original warning was added to the readme.

I agree with your point about the ALEZ auto updater, and considering we're rebuilding the ISO monthly, it's probably unnecessary having the auto-updater. It's potentially a security concern since it allows for behind-the-scenes updates do whatever the user is executing, and it's hard for them to inspect what is actually being run. Another option would be a prompt that checks if there is a more updated version and if there is, notifies the user so they can either clone it, or download a more recent ISO. @danboid was the one who added it, so would be best to get his feedback on the reason why it was originally set up.

danboid commented 5 years ago

My warning about ALEZ and containers probably only applies to running ALEZ in an unprivileged LXC/LXD container although I haven't tested running it in a privileged LXC/LXD container to verify it does work that way. They both default to using unpriv I think as its supposedly more secure. I will try manually building the ISO in a privileged LXD container later.

The auto-updater script was to ensure that that ALEZ wasn't running an outdated version of alez.sh but it predates John automating the ISO build process via Travis and Docker. I thought this was the best way to do it because ALEZ won't function without internet access anyway but maybe it can go now the ISO building is automated.

qdm12 commented 5 years ago

Cool, thanks for your answers.

What I'm thinking now is to check internet connectivity at the start of alez.sh by pinging https://wiki.archlinux.org first and show an error message if it fails.

What would be the advantage of using LXC/LXD instead of Docker?

Finally, do you think it's possible to build the ISO at the Docker build stage (so have a privileged build)? The run command could then just move the ISO to the bind mounted volume for the host. That could potentially allow to have a quick ISO rebuild when alez.sh changes, instead of having a whole rebuild.

danboid commented 5 years ago

What I'm thinking now is to check internet connectivity at the start of alez.sh by pinging https://wiki.archlinux.org first and show an error message if it fails.

A ping sounds like a good idea. It'd be better for it to ping the arch repos or the archzfs repo tho.

What would be the advantage of using LXC/LXD instead of Docker?

Docker is cross platform but LXC/LXD is easier to use to set up a (scratch) container for Linux users.

I'll let John reply to your other suggestion as the docker support is his work.

johnramsden commented 5 years ago

@qdm12 From what I understand privilege builds aren't supported - https://github.com/moby/moby/issues/1916

I just gave it a try and it won't work with the current build since it can't mount /proc, or at least that's where it first errors out.

I'm not sure it would make much of a difference in terms of time anyway, since once it's built, if there are no arch updates, a rebuild would be quite quick anyway just copying in alez.sh. In terms of Travis, the whole build would have to happen anyway, unless we were to pull from a cached docker image, so it wouldn't speed that up.

johnramsden commented 5 years ago

In regards to the removal of the auto updater, in order to still notify the user should we print a message when there is a newer version of the ISO?

I've got a rough draft of a system that checks the current latest tagged release, and compares it to the currently used alez.sh, if there's a newer version it says:

A newer version of ALEZ is available at https://github.com/danboid/ALEZ//releases/latest.

Is this a good idea? Or should we just assume the user knows to look for a newer version?

Thoughts @danboid @qdm12 ?

danboid commented 5 years ago

I honestly have no issues with the auto-updater as it is because in most cases there will be little to no difference in using a newer ISO unless the keys have expired but if you've already got this working pretty much I've got no objections to doing this.

qdm12 commented 5 years ago

@johnramsden if you have a draft, go for it. It's better to let the user know about an update and let him re-download the iso manually as the alez.sh might depend on the version of zfs bundled in the Iso for example.

qdm12 commented 5 years ago

Maybe we could have the docker run have a command flag --onlyalez for example, that would take the existing ISO in the directory /opt/alez/iso/out and simply update the alez.sh into the ISO image. I'll work on that soon. What do you think?

johnramsden commented 5 years ago

@qdm12 Considering alez.sh isn't updated all that often, and we can trigger a new-build with CI whenever it says I think it's more work at its worth. That said, if you think it can be done easily and, in a reliable way I don't see why not.

If you are thinking more for development testing, what I usually do is just copy paste my test alez.sh into a new file and run that.