Using an aptly published mirror to install a new system with debian-installer

pfelecan commented 6 years ago

Detailed Description

I'm trying to use an aptly published mirror to install a new system with a Debian installer, in my case debian-testing-amd64-netinst.iso.

When prompted for a mirror, I choose the local mirror obtained by publishing a snapshot. Given that the packages are now signed with my key, the installer fails with a message about being unable to verify the signature of the mirror's metadata.

How can I add my public key to the installer in order to being able to use packages from such a mirror when installing a new system with the Debian installer?

rhopfer commented 6 years ago

There are different solutions for your problem:

Build a custom installer that contains your signature, see https://wiki.debian.org/DebianInstaller/Build

Set APT::Get::AllowUnauthenticated yes using a preseed file

d-i     debian-installer/allow_unauthenticated  boolean true

Pull a custom keyring with preseed using early_command

d-i preseed/early_command string \
wget -O /usr/share/keyrings/archive.gpg http://SERVER_ADDRESS/trustedkeys.gpg

pfelecan commented 6 years ago

Thank you for your answer. When I'm in a hurry I can use 2 which, by the way, is what happens when, after selecting a mirror published by aptly, the installation give a message about not being able to verify the signature and, if you ignore the error, it proceeds as is the debian-installer/allow_unauthenticated was set to true. Otherwise, the third solution is what I use when doing a netinst.

pfelecan commented 6 years ago

I reopen this because the 3rd solution doesn't work: when the verification of InRelease is done, the error signaling that there no public key available appears even though the file is present in /usr/share/keyrings.

rhopfer commented 6 years ago

Make sure /usr/share/keyrings/archive.gpg is a keyring, not just a public key.

gpg --no-default-keyring --keyring DOCUMENT_ROOT/trustedkeys.gpg --import repository.key

pfelecan commented 6 years ago

Well, I succeeded in a way which is the opposite of what you mention. Let me explain:

trustdedkeys.gpg is a file managed by gpgv; neither gpg1 or gpg v2 manage this file; it contains allowed keys.
pubring.gpg contains the public keyring.

These 2 files are taken from my ~/.gnupg directory where my private key used to sign the repository is stored.

When I put the trustedkeys.gpg in /usr/share/keyrings/archive.gpg I have the aforementioned error.

When I put pubring.gpg in /usr/share/keyrings/archive.gpg the error does not appear and the installation proceeds nominally.

From my point of view this is to be expected as apt thus uses my public key.

What do you think?

rhopfer commented 5 years ago

Please note, that trustedkeys.gpg was just the name I used for the keyring and has nothing to do with a file possible located in ~/.gnupg. Also in my solution I fetch the keyring to /usr/share/keyrings/archive.gpg on the installer file system, not on the target.

pfelecan commented 5 years ago

You're right for the destination directory; I corrected my comment. In conclusion, we must put the public part of the key used to sign the published snapshot in the above mentioned directory during the early preseeding.

pfelecan commented 5 years ago

Having installed another system, I remarked the following subtlety: Installing the public keyring in the installer's /usr/share/keyrings as archive.gpg copies the keyring over the symbolic link pointing toward /usr/share/keyrings/debian-archive.gpg; but that doesn't solve the issue because when the preseed/early-command is executed, the target partition is not yet created. However, the installer complains about the non availability of the relevant key for the packages to be installed and, after investigation, it seems that it looks for the keyring in the target partition. The only solution that I found to circumvent this issue is to copy the /usr/share/keyrings/archive.gpg to /target/etc/apt/trusted.gpg.d and answer retry to the installer's question.

jirib commented 5 years ago

Having installed another system, I remarked the following subtlety: Installing the public keyring in the installer's /usr/share/keyrings as archive.gpg copies the keyring over the symbolic link pointing toward /usr/share/keyrings/debian-archive.gpg; but that doesn't solve the issue because when the preseed/early-command is executed, the target partition is not yet created. However, the installer complains about the non availability of the relevant key for the packages to be installed and, after investigation, it seems that it looks for the keyring in the target partition. The only solution that I found to circumvent this issue is to copy the /usr/share/keyrings/archive.gpg to /target/etc/apt/trusted.gpg.d and answer retry to the installer's question.

Any final solution?

ohmer1 commented 4 years ago

Hi,

I had some troubles when trying to use my aptly repository with a preseeded installer.

Copying the public key as /usr/share/keyrings/ubuntu-archive-keyring.gpg (or archive.gpg) doesn't work. My solution is to import my public key into a new keyring from another computer:

cat aptly-pubkey.asc | gpg --no-default-keyring --keyring aptly-trustedkeys.gpg --import

Then, to upload the newly created aptly-trustedkeys.gpg keyring to my public aptly directory.

Finally, in my preseed file, I defined this:

d-i preseed/early_command string \
wget -O /usr/share/keyrings/ubuntu-archive-keyring.gpg https://aptly.example.tld/aptly-trustedkeys.gpg

Seem to work fine. Hope that help!

alexises commented 4 years ago

Ok, I spend a whole day on this issue on my side.

To be short, on Debian 10 install the provider could not work.

Debian installation stage go as this :

Basic install
Apt package check
Installation of extra packages

Basic install

With a netinstall distk, the basic install stage use debootstrap.

This step of installation is performed using the CD content, so, no need for extra gpg key.

This basic stage will deploy package that provides the gpg key for the official repository, and so, on normal circustances, the followings steps will perform smoothly.

Apt package check (apt-setup module)

Here, the tool will deploy and check the sources.list config, with :

basic Debian repository
custom user configured repository

All routines are managed on a "generators" dir, executed one at a time. the 2 main files here are :

50mirror (set the main mirror)
60local (set extra mirrors)

Our issue is on 50mirror, due to a call of the apt-setup-verify routine to check if the provided apt configuration is valid.

Due to the disabling of skip-verify on apt with Debian 10, you can't now relay on configuration d-i debian-installer/allow_unauthenticated !

In addition, deboostrap is not used, and you can't use the host file in /usr/share/keyrings to validate the corresponding repository. Here is the corresponding line used to check the repo :

        $logoutput $intarget debconf-apt-progress --logstderr $dap_opts -- \
                apt-get -o APT::Get::List-Cleanup=false \
                        -o Dir::Etc::sourcelist=$tmp $ASV_TIMEOUT update || code=$?

No call to debootstrap, no way to add a gpg key.

Dirty hack (but with a loose of automation)

I only get one workaround for this :

on your preseed, install one extra repository and define the appropriate key that is common between the base repository and your custom one
performing installation until you got the repository error, click the ignore option for the repository
installation will continue and fail on next step,
revert to the apt configuration task. With the deployment of the key on /etc/apt/trusted.gpg.d you'll will be able to perform the installation normally

petrmalkov commented 2 years ago

my goal was to make my own ISO with new packs and key solution: copy your signing pub key into iso pool/extras/packages/domain.gpg.key escape using '@' in filename, it fails on script running

d-i partman/early_command string apt-install gnupg ; \
  echo -e "cp /cdrom/pool/extras/packages/domain.gpg.key /target/tmp/domain.gpg.key\nchroot /target /bin/bash /bin/apt-key add /tmp/domain.gpg.key" > /usr/lib/apt-setup/generators/001add-key ; \
  chmod +x /usr/lib/apt-setup/generators/001add-key

what's going on: 'partman/early_command' is used instead of 'preseed/early_command' - it is a step when all is prepared in memory and /target is going to be done /usr/lib/apt-setup/generators/001add-key is loading into memory and it will be executed on the stage of apt-setup just before packets installation

RolandKaechele commented 2 years ago

I have found a very simple solution for adding an aptly key to debian-installer for bullseye

Needed files https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/current/amd64/iso-cd/firmware-11.2.0-amd64-netinst.iso or https://cdimage.debian.org/cdimage/release/11.2.0/amd64/iso-cd/debian-11.2.0-amd64-netinst.iso

http://ftp.hosteurope.de/pub/linux/debian/indices/override.bullseye.main.gz

Create aptly-keys deb file

mkdir aptly-keys/aptly-keys-2022.1.0_all/{DEBIAN,etc/apt/trusted.gpg.d}
cat >> aptly-keys/aptly-keys-2022.1.0_all/DEBIAN/control << EOF
Package: aptly-keyring
Version: 2022.1.0
Architecture: all
Maintainer: Automatic Signing Key <ftp-master@example.org>
Section: misc
Priority: important
Multi-Arch: foreign
Essential: yes
Description: GnuPG archive keys for aptly
EOF

The field "Essential: yes" is important. It causes the debian-installer to install this deb before software will be installed from the aptly repository.

Extract the public key to aptly-keys/aptly-keys-2022.1.0_all/etc/trusted.gpg.d and generate the deb file with

cd aptly-keys; dpkg-deb --build aptly-keys-2022.1.0_all; cd..

Modify debian-installer iso You can use the official debian-installer image or the unofficial debian-installer image with firmware (which I have used).

mkdir net-inst-image

dd if=firmware-11.2.0-amd64-netinst.iso bs=1 count=432 of=isohdpfx.bin
xorriso -osirrox on -indev firmware-11.2.0-amd64-netinst.iso -extract / net-inst-image/

chmod +w -R net-inst-image/

mkdir net-inst-image/pool/main/a/aptly-keys
cp ./aptly-keys/aptly-keys-2022.1.0_all.deb net-inst-image/pool/main/a/aptly-keys/

Regenerate package archive

mkdir indices
gunzip override.bullseye.main.gz
cp override.bullseye.main indices/

cat >> config-deb << EOF
Dir {
 ArchiveDir "net-inst-image";
 OverrideDir "indices";
 CacheDir "indices";
};

TreeDefault {
 Directory "pool/";
};

BinDirectory "pool/main" {
 Packages "dists/bullseye/main/binary-amd64/Packages";
 BinOverride "override.bullseye.main";
 #ExtraOverride "override.extra";
};

Default {
 Packages {
  Extensions ".deb";
 };
};
EOF

apt-ftparchive generate config-deb

sed -i '/MD5Sum:/,$d' net-inst-image/dists/bullseye/Release 
apt-ftparchive release net-inst-image/dists/bullseye >> net-inst-image/dists/bullseye/Release
rm net-inst-image/dists/bullseye/main/binary-amd64/Packages

cd net-inst-image; md5sum `find ! -name "md5sum.txt" ! -path "./isolinux/*" -follow -type f` > md5sum.txt; cd ..

chmod -w -R net-inst-image/

Repack iso image

cp net-inst-image/.disk/mkisofs mkisofs.sh
chmod 775 mkisofs.sh

Adapt mkisofs.sh and remove the jigdo entities and some paths

xorriso -as mkisofs \
 -r -V 'Debian 11.2.0 amd64 n' \
 -o ./firmware-11.2.0-amd64-netinst-modified.iso \
 -J -J -joliet-long -cache-inodes \
 -isohybrid-mbr isohdpfx.bin \
 -b isolinux/isolinux.bin \
 -c isolinux/boot.cat \
 -boot-load-size 4 -boot-info-table -no-emul-boot \
 -eltorito-alt-boot \
 -e boot/grub/efi.img \
 -no-emul-boot -isohybrid-gpt-basdat -isohybrid-apm-hfsplus \
 net-inst-image

./mkisofs.sh

hxlhxl commented 12 months ago

my goal was to make my own ISO with new packs and key solution: copy your signing pub key into iso pool/extras/packages/domain.gpg.key escape using '@' in filename, it fails on script running
d-i partman/early_command string apt-install gnupg ; \
  echo -e "cp /cdrom/pool/extras/packages/domain.gpg.key /target/tmp/domain.gpg.key\nchroot /target /bin/bash /bin/apt-key add /tmp/domain.gpg.key" > /usr/lib/apt-setup/generators/001add-key ; \
  chmod +x /usr/lib/apt-setup/generators/001add-key
what's going on: 'partman/early_command' is used instead of 'preseed/early_command' - it is a step when all is prepared in memory and /target is going to be done /usr/lib/apt-setup/generators/001add-key is loading into memory and it will be executed on the stage of apt-setup just before packets installation

you saved my day! I slightly modified the commands because apt-key require gpg but not avaiable .here is my preseed

d-i partman/early_command string \
  apt-install gnupg ; \
  echo -e "cp /usr/share/keyrings/mykeyring.gpg /target/tmp/mykeyring.gpg\nchroot /target /bin/bash -c '/bin/cp /tmp/mykeyring.gpg /etc/apt/trusted.gpg.d/'" > /usr/lib/apt-setup/generators/001add-key ; \
  chmod +x /usr/lib/apt-setup/generators/001add-key

neolynx commented 3 months ago

thanks for sharing !

bmeyer71 commented 3 months ago

I've used what @hxlhxl and @petrmalkov have provided and set things up to download the gpg key from a local webserver since I didn't want to update the ISO further and made it a bit more flexible for my use.

I download the gpg key and then copy it to the target using:

d-i preseed/early_command string wget -O /tmp/apt_patching_pub.gpg http://repo.home.arpa/keys/apt_patching_pub.gpg; \
echo -e "cp /tmp/apt_patching_pub.gpg /target/tmp/apt_patching_pub.gpg\nchroot /target /bin/bash -c '/bin/cp /tmp/apt_patching_pub.gpg /etc/apt/trusted.gpg.d/'" > /usr/lib/apt-setup/generators/001add-key; \
chmod +x /usr/lib/apt-setup/generators/001add-key

This seems to be working well so far.

aptly-dev / aptly