copyfile source file collision

[tim-oe]

just installed the latest version using 2024-03-15-raspios-bookworm-arm64-lite.img building image on same version launched on a pi 3b+

tried with both the following:

separate plugins --plugin copyfile:"from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinstall|chown=tcronin:tcronin|chmod=600" \ --plugin copyfile:"from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinstall|chown=ansible:ansible|chmod=600" \

and filelist --plugin copyfile:"filelist=./filelist" \ filelist content from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|chown=ansible:ansible|chmod=600|mkdirif|runphase=postinstall from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=600|mkdirif|runphase=postinstall both instances would copy the first from to each to target even verified reordering the files also the filelist command required adding the mkdirif even with a preceding mkdir plugin creating the target dir.

full customize command: sudo sdm --customize \ --plugin user:"adduser=tcronin|password=$TEC_PWD" \ --plugin mkdir:"dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700" \ --plugin copyfile:"from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinstall|chown=tcronin:tcronin|chmod=600" \ --plugin user:"adduser=ansible|password=$ANS_PWD" \ --plugin mkdir:"dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700" \ --plugin copyfile:"from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinstall|chown=ansible:ansible|chmod=600" \ --plugin user:"deluser=pi" \ --plugin L10n:"keymap=us|locale=en_US.UTF-8|timezone=America/Chicago" \ --regen-ssh-host-keys \ --plugin network:"netman=nm|wifissid=tec-wan|wifipassword=$WIFI_PWD|wificountry=US" \ --restart img/bookworm.img

[gitbls]

Investigating...

[gitbls]

Please provide the redacted-as-necessary /etc/sdm/history from the IMG or console output from the customize command. Thx

[tim-oe]

I think i found the issue from command output: first copy: `> Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments: 'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 1

  Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/tcronin/.ssh' Plugin copyfile: Set owner 'tcronin:tcronin' on file '/home/tcronin/.ssh/authorized_keys' Plugin copyfile: Set mode '600' on file '/home/tcronin/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1 `

second copy: `> Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments: 'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 1

  Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/tcronin/.ssh' Plugin copyfile: Set owner 'tcronin:tcronin' on file '/home/tcronin/.ssh/authorized_keys' Plugin copyfile: Set mode '600' on file '/home/tcronin/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1

• `

key being: Copy 'authorized_keys' from /etc/sdm/assets/copyfile

below is the full redacted output:

tcronin@tec-pi-mgr:~/src/piImage $ src/sdm/image.sh

• Mount IMG 'img/bookworm.img' mount: /dev/loop0 mounted on /mnt/sdm. mount: /dev/loop1 mounted on /mnt/sdm/boot/firmware.
• Start Configuration

  Command Line: /usr/local/bin/sdm --customize --plugin user:adduser=tcronin|password= --plugin mkdir:dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700 --plugin copyfile:from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase= postinstall|chown=tcronin:tcronin|chmod=600 --plugin user:adduser=ansible|password= --plugin mkdir:dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700 --plugin copyfile:from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase= postinstall|chown=ansible:ansible|chmod=600 --plugin user:deluser=pi --plugin L10n:keymap=us|locale=en_US.UTF-8|timezone=America/Chicago --plugin disables:piwiz --regen-ssh-host-keys --plugin network:netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US --restart img/bookworm.img

• Host Information Hostname: tec-pi-mgr uname: Linux tec-pi-mgr 6.6.20+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.20-1+rpt1 (2024-03-07) aarch64 GNU/Linux os-release Name: Debian GNU/Linux 12 (bookworm) Version: 12 (bookworm) sdm Version: V11.6
• IMG Information Name: img/bookworm.img Date: 2024-03-15 RasPiOS Version: 12 RasPiOS Architecture: 64-bit aarch64 os-release Version: 12 (bookworm)
• Plugins selected:
  • user Args: adduser=tcronin|password=
  • mkdir Args: dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700
  • copyfile Args: from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinstall|chown=tcronin:tcronin|chmod=600
  • user Args: adduser=ansible|password=
  • mkdir Args: dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700
  • copyfile Args: from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinstall|chown=ansible:ansible|chmod=600
  • user Args: deluser=pi
  • L10n Args: keymap=us|locale=en_US.UTF-8|timezone=America/Chicago
  • disables Args: piwiz
  • network Args: netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US

    IMG 'img/bookworm.img' has 319788 1K-blocks (0.3GB, 0.3GiB) free Copy sdm to /usr/local/sdm in the IMG

• Start Phase 0 image customization

  Run Plugins Phase '0' Run Plugin 'user' (/mnt/sdm/usr/local/sdm/plugins/user) Phase 0 with arguments: 'adduser=tcronin|password='

• Plugin user: Start Phase 0

  Plugin user: Keys/values found: adduser: tcronin password: Plugin user: Create user 'tcronin' home directory '/home/tcronin' so available for other plugins

• Plugin user: Complete Phase 0

  Run Plugin 'mkdir' (/mnt/sdm/usr/local/sdm/plugins/mkdir) Phase 0 with arguments: 'dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700'

• Plugin mkdir: Start Phase 0

  Plugin mkdir: Keys/values found: dir: /home/tcronin/.ssh chown: tcronin:tcronin chmod: 700 Plugin mkdir: Create directory '/home/tcronin/.ssh'

• Plugin mkdir: Complete Phase 0

  Run Plugin 'copyfile' (/mnt/sdm/usr/local/sdm/plugins/copyfile) Phase 0 with arguments: 'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 0

  Plugin copyfile: Keys/values found: from: /mnt/clones/data/home/tcronin/.ssh/authorized_keys to: /home/tcronin/.ssh runphase: postinstall chown: tcronin:tcronin chmod: 600 Plugin copyfile: Add '/mnt/clones/data/home/tcronin/.ssh/authorized_keys' to files to copy

• Plugin copyfile: Complete Phase 0

  Run Plugin 'user' (/mnt/sdm/usr/local/sdm/plugins/user) Phase 0 with arguments: 'adduser=ansible|password='

• Plugin user: Start Phase 0

  Plugin user: Keys/values found: adduser: ansible password: Plugin user: Create user 'ansible' home directory '/home/ansible' so available for other plugins

• Plugin user: Complete Phase 0

  Run Plugin 'mkdir' (/mnt/sdm/usr/local/sdm/plugins/mkdir) Phase 0 with arguments: 'dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700'

• Plugin mkdir: Start Phase 0

  Plugin mkdir: Keys/values found: dir: /home/ansible/.ssh chown: ansible:ansible chmod: 700 Plugin mkdir: Create directory '/home/ansible/.ssh'

• Plugin mkdir: Complete Phase 0

  Run Plugin 'copyfile' (/mnt/sdm/usr/local/sdm/plugins/copyfile) Phase 0 with arguments: 'from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinst all|chown=ansible:ansible|chmod=600'

• Plugin copyfile: Start Phase 0

  Plugin copyfile: Keys/values found: from: /mnt/clones/data/home/ansible/.ssh/authorized_keys to: /home/ansible/.ssh runphase: postinstall chown: ansible:ansible chmod: 600 Plugin copyfile: Add '/mnt/clones/data/home/ansible/.ssh/authorized_keys' to files to copy

• Plugin copyfile: Complete Phase 0

  Run Plugin 'user' (/mnt/sdm/usr/local/sdm/plugins/user) Phase 0 with arguments: 'deluser=pi'

• Plugin user: Start Phase 0

  Plugin user: Keys/values found: deluser: pi

• Plugin user: Complete Phase 0

  Run Plugin 'L10n' (/mnt/sdm/usr/local/sdm/plugins/L10n) Phase 0 with arguments: 'keymap=us|locale=en_US.UTF-8|timezone=America/Chicago'

• Plugin L10n: Start Phase 0

  Plugin L10n: Keys/values found: keymap: us locale: en_US.UTF-8 timezone: America/Chicago Plugin L10n: Save L10n configuration

• Plugin L10n: Complete Phase 0

  Run Plugin 'disables' (/mnt/sdm/usr/local/sdm/plugins/disables) Phase 0 with arguments: 'piwiz'

• Plugin disables: Start Phase 0

  Plugin disables: Keys/values found: piwiz:

• Plugin disables: Complete Phase 0

  Run Plugin 'network' (/mnt/sdm/usr/local/sdm/plugins/network) Phase 0 with arguments: 'netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US' Plugin network: Configuring 'network-manager' for network configuration management % Plugin network: Network Manager requires either a .nmconnection file for WiFi or explicit WiFi configuration via arguments: wifissid wifipassword wificountry Found neither; WiFi likely will not connect

• Phase 0 Completed
• Enter image 'img/bookworm.img' for Phase 1
• Start Phase 1 image customization

  IMG 'img/bookworm.img' has 319696 1K-blocks (0.3GB, 0.3GiB) free at start of Phase 1 image customization Configure and enable sdm FirstBoot service (sdm-firstboot) Start 'apt update' IMG 'img/bookworm.img' has 319692 1K-blocks (0.3GB, 0.3GiB) free at start of 'apt update' IMG 'img/bookworm.img' has 205272 1K-blocks (0.2GB, 0.2GiB) free at end of 'apt update' Run Plugins Phase '1' Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase 1 with arguments: 'adduser=tcronin|password='

• Plugin user: Start Phase 1

  Plugin user: Add user 'tcronin' Plugin user: Add groups to user 'tcronin': dialout,cdrom,floppy,audio,video,plugdev,users,adm,sudo,users,input,netdev,spi,i2c,gpio Plugin user: Enable sudo for user 'tcronin' Plugin user: Copy skel files for user 'tcronin' Plugin user: Set user 'tcronin' home directory '/home/tcronin' owner to 'tcronin:tcronin'

• Plugin user: Complete Phase 1

  Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase 1 with arguments: 'dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700'

• Plugin mkdir: Start Phase 1
• Plugin mkdir: Complete Phase 1

  Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments: 'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase 1

  Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/tcronin/.ssh' Plugin copyfile: Set owner 'tcronin:tcronin' on file '/home/tcronin/.ssh/authorized_keys' Plugin copyfile: Set mode '600' on file '/home/tcronin/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1

  Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase 1 with arguments: 'adduser=ansible|password='

• Plugin user: Start Phase 1

  Plugin user: Add user 'ansible' Plugin user: Add groups to user 'ansible': dialout,cdrom,floppy,audio,video,plugdev,users,adm,sudo,users,input,netdev,spi,i2c,gpio Plugin user: Enable sudo for user 'ansible' Plugin user: Copy skel files for user 'ansible' Plugin user: Set user 'ansible' home directory '/home/ansible' owner to 'ansible:ansible'

• Plugin user: Complete Phase 1

  Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase 1 with arguments: 'dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700'

• Plugin mkdir: Start Phase 1
• Plugin mkdir: Complete Phase 1

  Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase 1 with arguments: 'from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinst all|chown=ansible:ansible|chmod=600'

• Plugin copyfile: Start Phase 1

  Plugin copyfile: Copy 'authorized_keys' from /etc/sdm/assets/copyfile to '/home/ansible/.ssh' Plugin copyfile: Set owner 'ansible:ansible' on file '/home/ansible/.ssh/authorized_keys' Plugin copyfile: Set mode '600' on file '/home/ansible/.ssh/authorized_keys'

• Plugin copyfile: Complete Phase 1

  Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase 1 with arguments: 'deluser=pi'

• Plugin user: Start Phase 1

  Plugin user: Delete user 'pi' userdel: pi mail spool (/var/mail/pi) not found

• Plugin user: Complete Phase 1

  Run Plugin 'L10n' (/usr/local/sdm/plugins/L10n) Phase 1 with arguments: 'keymap=us|locale=en_US.UTF-8|timezone=America/Chicago'

• Plugin L10n: Start Phase 1 % Plugin L10n: Keymap 'us' will be set during sdm FirstBoot

  Plugin L10n: Disable keyboard-setup service; will be re-enabled during sdm FirstBoot Removed "/etc/systemd/system/sysinit.target.wants/keyboard-setup.service". Plugin L10n: Set locale to 'en_US.UTF-8' Generating locales (this might take a while)... en_US.UTF-8... done Generation complete. Plugin L10n: Set timezone to 'America/Chicago'

• Plugin L10n: Complete Phase 1

  Run Plugin 'disables' (/usr/local/sdm/plugins/disables) Phase 1 with arguments: 'piwiz'

• Plugin disables: Start Phase 1
• Plugin disables: Complete Phase 1

  Run Plugin 'network' (/usr/local/sdm/plugins/network) Phase 1 with arguments: 'netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US'

• Plugin network: Start Phase 1
• Plugin network: Complete Phase 1
• Phase 1 post-app installation/configuration

  Start 'apt upgrade' IMG 'img/bookworm.img' has 200380 1K-blocks (0.2GB, 0.2GiB) free at start of 'apt upgrade' IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at end of 'apt upgrade' Start 'apt autoremove' IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at start of 'apt autoremove' IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at end of 'apt autoremove' IMG 'img/bookworm.img' has 196856 1K-blocks (0.2GB, 0.2GiB) free at end of Phase 1 image customization

• Phase 1 Completed

  Run Plugins Phase 'post-install' Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase post-install with arguments: 'adduser=tcronin|password='

• Plugin user: Start Phase post-install
• Plugin user: Complete Phase post-install

  Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase post-install with arguments: 'dir=/home/tcronin/.ssh|chown=tcronin:tcronin|chmod=700'

• Plugin mkdir: Start Phase post-install

  Plugin mkdir: Change owner on '/home/tcronin/.ssh' to 'tcronin:tcronin' Plugin mkdir: Change directory protection for '/home/tcronin/.ssh' to '700'

• Plugin mkdir: Complete Phase post-install

  Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase post-install with arguments: 'from=/mnt/clones/data/home/tcronin/.ssh/authorized_keys|to=/home/tcronin/.ssh|runphase=postinst all|chown=tcronin:tcronin|chmod=600'

• Plugin copyfile: Start Phase post-install
• Plugin copyfile: Complete Phase post-install

  Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase post-install with arguments: 'adduser=ansible|password='

• Plugin user: Start Phase post-install
• Plugin user: Complete Phase post-install

  Run Plugin 'mkdir' (/usr/local/sdm/plugins/mkdir) Phase post-install with arguments: 'dir=/home/ansible/.ssh|chown=ansible:ansible|chmod=700'

• Plugin mkdir: Start Phase post-install

  Plugin mkdir: Change owner on '/home/ansible/.ssh' to 'ansible:ansible' Plugin mkdir: Change directory protection for '/home/ansible/.ssh' to '700'

• Plugin mkdir: Complete Phase post-install

  Run Plugin 'copyfile' (/usr/local/sdm/plugins/copyfile) Phase post-install with arguments: 'from=/mnt/clones/data/home/ansible/.ssh/authorized_keys|to=/home/ansible/.ssh|runphase=postinst all|chown=ansible:ansible|chmod=600'

• Plugin copyfile: Start Phase post-install
• Plugin copyfile: Complete Phase post-install

  Run Plugin 'user' (/usr/local/sdm/plugins/user) Phase post-install with arguments: 'deluser=pi'

• Plugin user: Start Phase post-install
• Plugin user: Complete Phase post-install

  Run Plugin 'L10n' (/usr/local/sdm/plugins/L10n) Phase post-install with arguments: 'keymap=us|locale=en_US.UTF-8|timezone=America/Chicago' Run Plugin 'disables' (/usr/local/sdm/plugins/disables) Phase post-install with arguments: 'piwiz'

• Plugin disables: Start Phase post-install

  Plugin disables: Disable running piwiz and userconfig on first system boot

• Plugin disables: Complete Phase post-install

  Run Plugin 'network' (/usr/local/sdm/plugins/network) Phase post-install with arguments: 'netman=nm|wifissid=tec-wan|wifipassword=|wificountry=US'

• Plugin network: Start Phase post-install

  Plugin network: Keys/values found: netman: network-manager wifissid: tec-wan wifipassword: wificountry: US Plugin network: Set WiFi Country 'US' for sdm FirstBoot Plugin network: Enable SSH service via /boot/ssh

• Plugin network: Complete Phase post-install

  Run graphics post-install configuration Plugin sdm-phase1: No known Display Manager found Plugin sdm-phase1: Set RasPiOS to B1:'console no login' for sdm FirstBoot IMG 'img/bookworm.img' has 196848 1K-blocks (0.2GB, 0.2GiB) free at end of image customization

• Enter Shell Command Prompt 'exit' to exit back to host system

[gitbls]

Thanks! Will correct this for next release.

[gitbls]

This is fixed in v11.7. Please test/verify. Thx

[tim-oe]

works! thanks for the quick turn around.

[gitbls]

Perfect. And thanks for finding this rather nasty bug. Using the same source filename in two different directories never occurred to me. 😵