search

puppetlabs / puppetlabs-release

Repo that builds packages to add our apt and yum repositories and public signing key.
7 stars 39 forks source link

RE-16301: add /etc/apt/preferences.d/puppet-release.pref to .deb #307

Closed e-gris closed 2 months ago

e-gris commented 3 months ago

I needed to introduce a 'puppet_major_version' into the JSON configs to support this request.

Aside from that, it just meant adding a new template for the .deb packages.

shaigy commented 3 months ago

LGTM.

joshcooper commented 3 months ago

hmm it does seem to work for Debian 12: 

root@coy-bromphenol:~# dpkg -i puppet7-release/output/deb/focal/puppet7/noarch/puppet7-release_7.0.0-18focal_all.deb 
root@coy-bromphenol:~# curl -LO http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repo_configs/deb/pl-puppet-agent-8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe-bookworm.list
root@coy-bromphenol:~# cp pl-puppet-agent-8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe-bookworm.list  /etc/apt/sources.list.d/puppet7-release.list 
root@coy-bromphenol:~# vi /etc/apt/sources.list.d/puppet7-release.list
root@coy-bromphenol:~# cat /etc/apt/sources.list.d/puppet7-release.list
# Packages for puppet-agent built from ref 8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe
deb [trusted=yes] http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm puppet7
root@coy-bromphenol:~# apt-get update
Ign:1 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm InRelease
Get:2 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm Release [8,367 B]
Ign:3 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm Release.gpg
Get:4 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 Packages [4,822 B]
Hit:5 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm InRelease
Hit:6 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm-updates InRelease
Hit:7 https://artifactory.delivery.puppetlabs.net/artifactory/debian_security__remote bookworm-security/updates InRelease
Fetched 13.2 kB in 1s (20.8 kB/s)
Reading package lists... Done
N: Repository 'Debian bookworm' changed its 'non-free component' value from 'non-free' to 'non-free non-free-firmware'
N: More information about this can be found online in the Release notes at: https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.html#non-free-split
root@coy-bromphenol:~# apt-cache policy puppet-agent
puppet-agent:
  Installed: (none)
  Candidate: 7.30.0.77.g8be6f291e-1bookworm
  Version table:
     7.30.0.77.g8be6f291e-1bookworm 1001
        500 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 Packages
     7.23.0-1 1001
        500 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm/main amd64 Packages
root@coy-bromphenol:~# apt-get install puppet-agent
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  puppet-agent
0 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
Need to get 31.1 MB of archives.
After this operation, 110 MB of additional disk space will be used.
Get:1 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 puppet-agent amd64 7.30.0.77.g8be6f291e-1bookworm [31.1 MB]
Fetched 31.1 MB in 0s (114 MB/s)  
Selecting previously unselected package puppet-agent.
(Reading database ... 40682 files and directories currently installed.)
Preparing to unpack .../puppet-agent_7.30.0.77.g8be6f291e-1bookworm_amd64.deb ...
Unpacking puppet-agent (7.30.0.77.g8be6f291e-1bookworm) ...
Setting up puppet-agent (7.30.0.77.g8be6f291e-1bookworm) ...
Created symlink /etc/systemd/system/multi-user.target.wants/pxp-agent.service → /lib/systemd/system/pxp-agent.service.
Created symlink /etc/systemd/system/multi-user.target.wants/puppet.service → /lib/systemd/system/puppet.service.
Removed "/etc/systemd/system/multi-user.target.wants/pxp-agent.service".
Processing triggers for libc-bin (2.36-9+deb12u4) ...
joshcooper commented 3 months ago

@e-gris from the apt-cache policy puppet-agent output, it seems both repos have priority 1001. Is there a way for our preferences file to be more specific so it only matches "our" repo?

  Version table:
     7.30.0.77.g8be6f291e-1bookworm 1001
        500 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 Packages
     7.23.0-1 1001
        500 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm/main amd64 Packages
e-gris commented 3 months ago

@e-gris from the apt-cache policy puppet-agent output, it seems both repos have priority 1001. Is there a way for our preferences file to be more specific so it only matches "our" repo?

  Version table:
     7.30.0.77.g8be6f291e-1bookworm 1001
        500 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 Packages
     7.23.0-1 1001
        500 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm/main amd64 Packages

I'll start digging into it. By "our" repository, this means "apt.puppet(labs).com", correct?

joshcooper commented 3 months ago

this means "apt.puppet(labs).com", correct?

Yeah though more specifically I was hoping it would only match the name of the repo that was installed by:

# cat /etc/apt/sources.list.d/puppet7-release.list 
# Packages for puppet-agent built from ref 8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe
deb [trusted=yes] http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm puppet7

so in this case the puppet7 repo (which might point to apt.p.com or builds.d.pl.net or nightlies.p.com)

e-gris commented 3 months ago

Hmm. Absolutely worthy of investigation but I'm not feeling optimistic.

e-gris commented 3 months ago

@joshcooper I think this might do the trick. Can you test against your cases?

Package: puppet-agent
Pin: release o=Puppetlabs
Pin-Priority: 1001
joshcooper commented 3 months ago

@e-gris ah using o=puppetlabs doesn't work:

# cat /etc/apt/sources.list.d/puppet7-release.list 
# Packages for puppet-agent built from ref 8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe
deb [trusted=yes] http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm puppet7
root@coy-bromphenol:~# cat /etc/apt/preferences.d/puppet-release.pref 
Package: puppet-agent
Pin: release o=Puppetlabs
Pin-Priority: 1001
# apt-get update
...
# apt-cache policy puppet-agent
puppet-agent:
  Installed: 7.30.0.77.g8be6f291e-1bookworm
  Candidate: 7.30.0.77.g8be6f291e-1bookworm
  Version table:
 *** 7.30.0.77.g8be6f291e-1bookworm 500
        500 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 Packages
        100 /var/lib/dpkg/status
     7.23.0-1 500
        500 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm/main amd64 Packages

But using o=Puppet Labs does:

# cat /etc/apt/preferences.d/puppet-release.pref 
Package: puppet-agent
Pin: release o=Puppet Labs
Pin-Priority: 1001
# apt-get update
...
# apt-cache policy puppet-agent
puppet-agent:
  Installed: (none)
  Candidate: 7.30.0.77.g8be6f291e-1bookworm
  Version table:
     7.30.0.77.g8be6f291e-1bookworm 1001
        500 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 Packages
        100 /var/lib/dpkg/status
     7.23.0-1 500
        500 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm/main amd64 Packages

Not sure if Debian bookworm is different than the other deb repos, but this is what I see:

# apt-cache policy
Package files:
...
 500 http://builds.delivery.puppetlabs.net/puppet-agent/8be6f291e0e24ece1d7b64dd30a344f0bedfb8fe/repos/apt/bookworm bookworm/puppet7 amd64 Packages
     release o=Puppet Labs,n=bookworm,l=Puppet Labs,c=puppet7,b=amd64
     origin builds.delivery.puppetlabs.net
...
e-gris commented 3 months ago

o=Puppet Labs does not work in my case but o=Puppetlabs does

With o=Puppet Labs:

oot@heroic-pacifism:~# apt-cache policy puppet-agent
puppet-agent:
  Installed: (none)
  Candidate: 7.30.0-1bookworm
  Version table:
     7.30.0-1bookworm 500
        500 http://apt.puppet.com bookworm/puppet7 amd64 Packages
     7.29.1-1bookworm 500
        500 http://apt.puppet.com bookworm/puppet7 amd64 Packages
     7.29.0-1bookworm 500
        500 http://apt.puppet.com bookworm/puppet7 amd64 Packages
     7.23.0-1 500
        500 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm/main amd64 Packages

Changing it to o=Puppetlabs

root@heroic-pacifism:~# apt-cache policy puppet-agent
puppet-agent:
  Installed: (none)
  Candidate: 7.30.0-1bookworm
  Version table:
     7.30.0-1bookworm 1001
        500 http://apt.puppet.com bookworm/puppet7 amd64 Packages
     7.29.1-1bookworm 1001
        500 http://apt.puppet.com bookworm/puppet7 amd64 Packages
     7.29.0-1bookworm 1001
        500 http://apt.puppet.com bookworm/puppet7 amd64 Packages
     7.23.0-1 500
        500 https://artifactory.delivery.puppetlabs.net/artifactory/debian__remote bookworm/main amd64 Packages
e-gris commented 3 months ago

My test is on Debian 12

I'll see if I can hunt down the source of the inconsistency, right now its looking like something in the Debian distro itself, which is fun.

e-gris commented 3 months ago

This is getting stranger. The only difference I'm seeing right now is in the .list file

root@heroic-pacifism:~# cat /etc/apt/sources.list.d/puppet7-release.list 
# Puppet 7 bookworm Repository
deb http://apt.puppet.com bookworm puppet7

This indicates to me that the Puppet Labs v. Puppetlabs is baked into the repo at http://builds.delivery.puppetlabs.net and http://apt.puppet.com

e-gris commented 3 months ago

I'm thinking maybe:

root@young-repeat:~# cat /etc/apt/preferences.d/puppet-release.pref 
Package: puppet-agent
Pin: release o=Puppet*
Pin-Priority: 1001
joshcooper commented 3 months ago

Ah o=Puppet* works for me when using builds.d.p.net as in our dev version 7.30.0.81 has a higher priority than upstream 8.4.0:

root@upriver-adrenal:~# cat /etc/apt/preferences.d/puppet-release.pref 
Package: puppet-agent
Pin: release o=Puppet*
Pin-Priority: 1001
root@upriver-adrenal:~# cat /etc/apt/sources.list.d/puppet7-release.list 
# Packages for puppet-agent built from ref b9bc7e363aa280def76e3917ed21ef75f2fee6cb
deb [trusted=yes] http://builds.delivery.puppetlabs.net/puppet-agent/b9bc7e363aa280def76e3917ed21ef75f2fee6cb/repos/apt/noble noble puppet7
root@upriver-adrenal:~# apt-cache policy puppet-agent
puppet-agent:
  Installed: (none)
  Candidate: 7.30.0.81.gb9bc7e363-1noble
  Version table:
     8.4.0-1 500
        500 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote noble/universe amd64 Packages
     7.30.0.81.gb9bc7e363-1noble 1001
        500 http://builds.delivery.puppetlabs.net/puppet-agent/b9bc7e363aa280def76e3917ed21ef75f2fee6cb/repos/apt/noble noble/puppet7 amd64 Packages

Does that mean we can get rid of the PUPPET_MAJOR_VERSION templating?

e-gris commented 3 months ago

Does that mean we can get rid of the PUPPET_MAJOR_VERSION templating?

Yes, it does.

skyamgarp commented 2 months ago

Hello @e-gris , When can we expect to merge this PR? @joshcooper , We would need to test this against Ubuntu 24 once the agent is ready?

e-gris commented 2 months ago

I'd like to see a final review from @joshcooper, at least. After that I feel it is good to go.