Using of sdm without of system-wide installation

[ykharuzhy]

Is it possible to use sdm without of installation to system-wide directories like /etc/sdm? This would be a good option to allow the user to build the image without installing additional software.

[gitbls]

No, it's not possible, and it's a huge amount of work and testing, and the value of it is completely unclear to me.

I don't understand your point about "This would be a good option to allow the user to build the image without installing additional software.".

What additional software are you referring to, on the host, or in the IMG? and how would you envision this being "better" in some meaningful way than the current design?

Thx

[ykharuzhy]

Hi, I want to deliver the repository with scripts needed to rebuild a customized image to customer. So, I prefer to avoid installing of sdm because it is not managed by system package manager and requires some manual steps. All other software packages can be installed from package repos.

But anyway, thank you for this great swiss-knife for image customization, I can't imagine how much time I would spend implementing a similar tool.

[gitbls]

OK, I understand what you're trying to do, but unfortunately it's a massive lift to eliminate the use of /etc/sdm.

One alternative would be for you to create your own repository to provide a .deb packaged version of sdm. I'm happy to work with you on this for the sdm side of things, but at the moment I'm not able to undertake this myself.

Additional thought: If you'd like to discuss privately, please feel free to email me. My email is on the sdm README.

[thk686]

I have created a Dockerfile that installs sdm in a container.

# Use the official Debian stable slim image
FROM debian:stable-slim

# Make sure we're up to date
RUN apt-get update && apt-get -y full-upgrade

# Install necessary packages
RUN apt-get update && apt-get install -y \
    sudo \
    curl \
    unzip \
    fdisk \
    dosfstools \
    rsync \
    parted \
    kpartx \
    qemu-utils \
    qemu-user-static \
    qemu-system-arm \
    qemu-efi-aarch64 \
    ipxe-qemu \
    qemu-efi-arm \
    qemu-system-gui \
    systemd-container \
    binfmt-support \
    util-linux \
    xz-utils \
    zip \
    bzip2 \
    file \
    less \
    && rm -rf /var/lib/apt/lists/*

# Create a new user 'agent' and set a password
RUN useradd -m agent && echo "agent:agent" | chpasswd

# Add the new user to the sudo group
RUN usermod -aG sudo agent

# Allow members of the sudo group to execute any command without a password
RUN echo "%sudo ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

# Switch to the new user and set the working directory
USER agent
WORKDIR /home/agent

RUN mkdir /home/agent/hostdir

# Install sdm utility
RUN sudo curl -L https://raw.githubusercontent.com/gitbls/sdm/master/EZsdmInstaller | sudo bash

COPY my-custom-plugin /usr/local/sdm/local-plugins
RUN sudo chmod +x /usr/local/sdm/local-plugins/my-custom-plugin

# Example command to run after verification (modify as needed)
CMD ["bash"]

Here is an example invocation:

docker run --privileged -it --rm -v .:/home/agent/hostdir -v /dev:/dev "$DOCKER_IMAGE_NAME" /bin/bash -v -c "
sudo sdm \
    --plugin @/home/agent/hostdir/sdm-plugin-commands \
    --plugin-debug \
    --autologin \
    --expand-root \
    --regen-ssh-host-keys \
    --hostname $IMAGE_HOSTNAME \
    --burnfile $CONTAINER_CUSTOMIZED_IMAGE_FILE \
    $CONTAINER_IMAGE_FILE

[mattie47]

Hi @thk686 and @simlu

Between the two of you (Simlu, on your https://github.com/gitbls/sdm/issues/155 thread), I wanted to confirm with the docker built scripts you've written, whether they actually work?

It wasn't clear to me from the above, or https://github.com/gitbls/sdm/issues/155 whether you'd successfully built an image, and had it output the img correctly from Docker?

i.e. are there still outstanding issues for you both?

Thanks!

[simlu]

Hello @mattie47

Something worked, but we ended up rolling our own (for better control). So at this point we are no longer using sdm and I won't be of much help unfortunately. I'm happy to share pieces of our script though if you have concrete questions!

Cheers, L~

[thk686]

I have a working setup that installs sdm in a docker container and then runs sdm to customize an image. I have not yet opened the repo, but will do so soon.

On Wed, Jul 10, 2024 at 11:23 PM Matt @.***> wrote:

Hi @thk686 https://github.com/thk686 and @simlu https://github.com/simlu

Between the two of you (Simlu, on your #155 https://github.com/gitbls/sdm/issues/155 thread), I wanted to confirm with the docker built scripts you've written, whether they actually work?

It wasn't clear to me from the above, or #155 https://github.com/gitbls/sdm/issues/155 whether you'd successfully built an image, and had it output the img correctly from Docker?

i.e. are there still outstanding issues for you both?

Thanks!

— Reply to this email directly, view it on GitHub https://github.com/gitbls/sdm/issues/216#issuecomment-2221998034, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEQXSOODHE2UIU4QA2XPRLZLYCEDAVCNFSM6AAAAABIYEQ4YGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRRHE4TQMBTGQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Timothy H. Keitt www keittlab org

[thk686]

Oof. If I had known about https://github.com/RPi-Distro/pi-gen I would have started there. Not as featured as sdm however.

[gitbls]

Oof. If I had known about https://github.com/RPi-Distro/pi-gen I would have started there. Not as featured as sdm however.

Pretty much my summary of it: https://github.com/gitbls/sdm/discussions/46

[mattie47]

Thanks @simlu, @thk686 for your response. It's appreciated.

I'm curious what you ended up going with as an alternative @simlu?

I used SDM last year to build a number of RPI images and found it to be fantastic. @gitbls, your commitment to helping others and continued support/documentation of this tool is what makes it so useful.

My problem now with ~20x RPIs is long maintenance.

I didn't realize until Bookworm came out that there's not an easy upgrade path for Raspi OS say compared to Ubuntu with dist-upgrade, and I can't just reflash the SDs cards in these Pis.

I'm then also looking at OTA flashing of images with perhaps an AB model using something like Mender.

It all feels like it becomes complicated quickly lol

[gitbls]

You might want to read this: https://github.com/gitbls/sdm/issues/237#issuecomment-2227504480 I describe how I manage my Pi "fleet". Not that it's perfect, but I do now have the ability to build all my Pi disks from a common "workstation" base, and apply per-specific configuration at burn time.

This enables me to wholesale upgrade systems by dropping in a new disk, just like the old system, but with fresh new bits.

Also, it is possible to upgrade the OS on a disk using some apt magic, but that same magic sometimes has problematic side effects that can leave your system not working well if you don't know what you're doing.

Software is complicated. There are usually solutions for any problem, including the basic problem you're raisin. However, there are risks and costs associated with each approach. Kind of a "learn as much as you can about it, and then choose your approach"

[odi89]

Hi @thk686 and @simlu

Between the two of you (Simlu, on your #155 thread), I wanted to confirm with the docker built scripts you've written, whether they actually work?

It wasn't clear to me from the above, or #155 whether you'd successfully built an image, and had it output the img correctly from Docker?

i.e. are there still outstanding issues for you both?

Thanks!

@gitbls

I can confirm that 90% is working when running inside docker, @thk686 solution is perfect for my usecase since i dont need to actually burn the images (Burning image to sd card is problematic if macos is your host system, the mounting issue on macos did send me down a long rabbit hole )

Else i want to congratulate @gitbls on beeing a wholesome maintainer and i encouruge everyone to give him the props he deserves 💸❤️