Closed sraver closed 1 month ago
Glad that you're finding sdm useful! Here's what is happening, I'll leave it to you to decide your path forward.
shadow
, and it's a system group (In Linux, group IDs less than 1000 are typically reserved for system groups).useradd
command to create the user, which also tries to create a group with the same name. Unfortunately, there's already a group named shadow
, so the useradd
command fails:
Plugin user: Add user 'shadow' useradd: group shadow exists - if you want to add this user to that group, use -g. ? Plugin user: useradd returned an error
shadow
was not created, all subsequent attempts that try to use the user shadow
also fail: specifically, your mkdir
and copyfile
plugin invocationsThe easiest fix for you is to rename the user shadow
to something else. At the moment there's no workaround in the user
plugin for this.
Here are some other notes and comments that I hope are helpful.
I'd encourage you to consider moving all your plugins into a pluglist file documented here. It eliminates clutter on the command line, makes it easier to see what plugins you're using, and you don't need all the double quotes around plugin arguments. So, your customize command line would be
sudo sdm --customize --plugin @/path/to/pluglist --host pearl 2024-07-04-raspios-bookworm-arm64-lite.img`
And /path/to/pluglist file would contain (I used xshadow
instead of shadow
):
L10n:host
user:adduser=xshadow
mkdir:dir=/home/xshadow/.ssh|chown=xshadow:xshadow|chmod=700
copyfile:from=authorized_keys|to=/home/xshadow/.ssh|chown=xshadow:xshadow|chmod=600|mkdirif
hotspot:config=hotspot.cfg
I don't know about you, but I think this is a lot easier to work with! [NOTE: I did not test the above]
I typically don't set the hostname during a customize, saving that until I burn the disk. That way, a customized IMG can be used for multiple target systems. Of course, you can still set the hostname on a burn if you've set it during customization, so not a problem, but I find that being rigorously prescriptive about the details keeps me from confusing myself 😉
I haven't looked at the hotspot
plugin in quite a while. LMK if you run into any problems with it.
Oh man! Good catch! When I was writing the name I thought.. shadow is too common of a name, thinking of /etc/shadow
, but I just went on with it ^_^ I wasn't aware of the existing group.
It works now! Thanks!
For organizing the plugins/commands I'm moving to the script ezsdm
, so I can have different folders for each card, with the script ready to be run, with some assets that might be necessary for each set up. I also find that having it on a list is way more managable than on cluttered commands, and easier to version :)
Good tip also on the hostname :+1: , I'm still figuring the commands and how they interact on the different phases. Jumping soon on the code to understand some internals I'm still missing :p
I love it!! It's prob becoming my default setup from now :)
Thanks!!
Closing the issue, since it was never such a thing :pray:
Oh man! Good catch! When I was writing the name I thought.. shadow is too common of a name, thinking of
/etc/shadow
, but I just went on with it ^_^ I wasn't aware of the existing group.It works now! Thanks!
Excellent ;) Thanks for including /etc/sdm/history, that makes it SO easy to see what's going on. I've been reading them for a few years so very good at scanning for problems.
For organizing the plugins/commands I'm moving to the script
ezsdm
, so I can have different folders for each card, with the script ready to be run, with some assets that might be necessary for each set up. I also find that having it on a list is way more managable than on cluttered commands, and easier to version :)Great! You're headed down the right path!
Good tip also on the hostname 👍 , I'm still figuring the commands and how they interact on the different phases. Jumping soon on the code to understand some internals I'm still missing :p
I love it!! It's prob becoming my default setup from now :)
Thanks!! Thanks for the positive feedback! Happy to help with any further questions, problems, suggestions, etc.
Thanks! :pray:
I think I'm starting to figure out the best way to use the tool. In fact most of my cards share some base set up, like hotspot, aliases, etc.
So I think having an ezsdm
for the base image makes sense. Then, specific projects can start from there. But this is where I'm not so sure how to proceed.
Can I run another --customize
on top of an already customized image? I assume the last customize will override something that was set on the base if I specify so, but keeping the rest intact. Is that a valid approach?
The thing is, for more complex cards, I think I'll need to set them up manually with --explore
. In this case the project folder will actually not hold its own ezsdm
, but just an already customized base image.
I would like to be able to back up everything with only the script, without having to push the *.img
's, but in the approach I explained above the "manual steps" would be lost.
It is still light-years better than what I have currently tho, which is some notes and bad memory :joy:
PS: I'm thinking it would be crazy dope to be able to connect Ansible playbooks with sdm, ultimate combo :rocket: Probably doesn't make sense to make any improvements on sdm for that tho, once the pi is up I could run the playbooks through the hotspot... hm... looking at that :)
@sraver some answers and a bit about what I do.
Yes, you can run another --customize
on an IMG, but I think there are better ways. That said, when you try to customize an already-customized IMG, sdm warns you. You can omit the warning by using --redo-customize
.
I try to script everything, so none of my systems are done manually. At the moment I have 3 Pi "servers":
apt-cacher-ng
. BTW, if you have multiple Pi systems and aren't using apt-cacher-ng, you should make getting that working a top priority. You're welcome in advance 😉My other Pi systems are all configured as workstations.
(There's also a X64 NUC10-based file server and VirtualBox host that I hate having to upgrade it (every new Debian release) because it's such a pain without sdm 🤣. And I always do a fresh install. Only the best fresh new bugs for MY systems 🤣, followed by 2 years of basically not futzing with it)
I have a "standard" configuration for my Pis. It's the "workstation" configuration. When I want to build a new server, regardless of what it is, I burn the disk for that server and run whatever plugins are needed to turn it into the server. For instance, to rebuild my DHCP/DNS server I run plugins to:
ndm
plugin)apt-cacher-ng
using the like-named plugin.nmconnection
file using the network
plugin that configures it's static IP address (kinda needed for the DHCP/DNS server)system
plugin to configure /etc/exportsThe burn command. $dev, $host, and $img are appropriately set.
sdm --burn $dev --hostname $host --expand-root --plugin @/l/work/mystuff/$host/$host.plugins $img
I have similar pluglist files ($host.plugins) for each server.
I'd argue that there's very little, if anything that you can only do manually by exploring into an IMG. It's taken me a while to get to this point, of course. For Bullseye, I was partially there. I spent some time completely redoing the implementation for my servers to take advantage of pluglists, new plugins, and new system capabilities, such as NetworkManager. I think it's much easier to achieve this nirvana with the current sdm and Bookworm.
The beauty of this, which I'm sure you recognize, is that bad memory is not a failure mode. Bad code might be 🤣 but I try to avoid that.
As far as backups, I don't do full backups. I figure out what config files I need to backup, and only back them up if they've changed. I have a script that runs nightly to do that.
re Ansible: I used it for a while, but it seemed fairly heavyweight and sluggish for my impatience to tolerate. That said, I see the value in it. If I were to use Ansible now, I would use sdm to install whatever bits needed to be installed on every system in my workstation IMG (ssh keys, etc). I'd install Ansible itself on whatever systems I want to be able to run ansible commands on, and I'd install the configuration data on an NFS share so that I can run ansible on any server as needed.
That's it in a nutshell. You'll undoubtedly do things differently than me.
TLDR: Figure out how to configure each and every system completely using sdm, and use as few customized IMGs as possible. Defer system-specific customization to when you burn the disk for that system.
Thanks for such detailed explanation!
Very helpful to get a glimpse of how you are using it, it helps understanding the thought process on the tool itself.
You opened a new rabbit hole for me now, ndm xD Previosuly I had been using pi-hole, but I'm in the search of something more lightweight and easily manageble, from the terminal if possible, and ndm seems to tick all the boxes!
I'm a feeling a bit stuck right now tho, not because any issue I'm finding in the tool, but more because of my lack my understanding of some pieces of the whole system I'm trying to implement.
Would you mind me writing an email to you to explain what I'm trying to achieve, and maybe I could get your thoughts on it? I don't mind sharing the final approach here, even though it would be completely off-topic on this very issue, but I feel I'm a bit lost, and I'd rather share a clean approach once I have it.
Regardless, thanks for your time, and all the tips and tricks! :heart:
PS: One related question! All the ndm
set up process, I'm guessing this is something that you do once the system is running, which makes it "volatile", in the sense that it's not backed up, right?
But I guess that's not really an issue, since configuration for ndm seems quite lightweight to set up, and also you are porbably not setting up this box very often, more likely is kind of a set-and-forget one, right?
ndm: yep, it does dhcp and dns really well, and it's lightweight. pihole has some nice features, but I've seen enough people having problems with it...
re ndm and backups: Correct, you still need to back up the configuration, but it's exactly one (json) file. With that you can easily rebuild and reinstall the dhcp and dns servers configs with 2 ndm commands. The json file is one of the files that I back up nightly, BTW, and my network doesn't change very often so once a day is fine. If you're changing the network a lot, you can back it up more often.
re email: 👍
Hi!
First of all, this is an amazing tool! Thanks for the work you've done on it!
I've been setting up my Pi's for a while manually, and as you know it becomes a pain. Yesterday doing some research I discovered
sdm
, and here I am learning about it, to automate all the systems I use.In this example I'm trying to set up a basic system with a hotspot, and that part I have it working already.
The issue comes when I try to do it under a different user than the default
pi
.Reading the logs, I finally noticed the error:
Looking for related issue I found #126 , so I tried changing
shadow:shadow
toshadow.shadow
, and later on also to onlyshadow
, without specifying the group.In all cases is the same outcome,
Invalid user
.I don't know if I might be doing something wrong, or if it's actually an issue, so I decided to open this one.
The customize command I'm running is:
And the trace of the command is:
Do you have any idea what could be?
Again, with
pi
user I get it to work, because the user obv is already there, but it would be cool to be able to set others :)Thanks!!