Custom "internal" software repos?

[TJM]

We are thinking about making a feature to enable custom "internal" software repositories. Some environments are "air gapped" (as in no Internet access), and as such, the default software repositories are not useful. I am trying to decide whether it would be smarter to use the "file" provisioner to just dump the contents of a directory into yum.repos.d or whether it should be some sort of scripted interface.

I am just throwing this out there to see if anyone else has already done this, and/or if anyone has any ideas about what would be good or bad?

Tommy

[TJM]

I found something in the centos5 files that handles this. However, I think replacing the repo files should be "optional" and it should also be more "robust" (not hardcoded to just these few files).

    {
      "destination": "/tmp",
      "source": "kickstart/centos5/yum.repos.d/",
      "type": "file"
    },
    {
      "execute_command": "echo 'vagrant' | {{.Vars}} sudo -E -S bash '{{.Path}}'",
      "inline": [
        "mv -f /tmp/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo",
        "mv -f /tmp/CentOS-fasttrack.repo /etc/yum.repos.d/CentOS-fasttrack.repo",
        "mv -f /tmp/CentOS-Sources.repo /etc/yum.repos.d/CentOS-Sources.repo",
        "mv -f /tmp/CentOS-Vault.repo /etc/yum.repos.d/CentOS-Vault.repo"
      ],
      "type": "shell"
    },

[boxcutter-robot]

The reason why it’s done is centos5 is for another purpose - CentOS 5 is end-of-life and the repo files have been moved to the vault because there are no longer updates for that. It’s not possible to install packages on CentOS 5 without changing these values. It can’t be optional as it won’t work (and the CentOS folks intentionally make this a speedbump for CentOS installs).

What I do for air-gapped installs and local mirrors of repos is just to use a “real” configuration management system to make the repo file changes robust like Ansible, Chef, Puppet, Saltstack, etc. rather than using bash scripts. It’s far easier to do that with a config management system - just add another step in the provisioning with packer.

These are intended to be “JEOS boxes” - “just enough OS” for a configuration management system. It is still intended that the heavy lifting for the OS install be done with more robust tooling than bash scripts. These are only intended to provide a very basic base setup that gets the OS bootstrapped with virtualization drivers.

On Jun 14, 2018, at 7:28 AM, Tommy McNeely notifications@github.com wrote:

I found something in the centos5 files that handles this. However, I think replacing the repo files should be "optional" and it should also be more "robust" (not hardcoded to just these few files).

{
  "destination": "/tmp",
  "source": "kickstart/centos5/yum.repos.d/",
  "type": "file"
},
{
  "execute_command": "echo 'vagrant' | {{.Vars}} sudo -E -S bash '{{.Path}}'",
  "inline": [
    "mv -f /tmp/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo",
    "mv -f /tmp/CentOS-fasttrack.repo /etc/yum.repos.d/CentOS-fasttrack.repo",
    "mv -f /tmp/CentOS-Sources.repo /etc/yum.repos.d/CentOS-Sources.repo",
    "mv -f /tmp/CentOS-Vault.repo /etc/yum.repos.d/CentOS-Vault.repo"
  ],
  "type": "shell"
},

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/boxcutter/centos/issues/81#issuecomment-397315926, or mute the thread https://github.com/notifications/unsubscribe-auth/AL5WAzr8QC1D4uv8mmNz9bpMCDrjGaKOks5t8nMNgaJpZM4Un8lw.

[TJM]

Thanks @boxcutter-robot, (you might be logged in with the wrong account?)

We are "contemplating" whether we should produce an "updated" box, beyond the 7.x release (quarterly, or maybe monthly?). However, in order to take advantage of the "yum update" capability, we would have to point the "repos" to our internal "mirror" (pulp). We do use a configuration management system once the box is provisioned, but I was trying to avoid that to create the base box (JEOS) :)

Ignoring the "yum install open-vm-tools" (which we could just add to the kickstart), the software installation for this box is sourced entirely from the ISOs during the kickstart process, so on virtualbox it mostly seems to work in our environment already. :) Now, its just about updates.

[TJM]

Since it doesn't look like anyone else has posted any ideas here, I am going to close this out. IF someone else is looking for something like this, we chose to add the following to the start of the provisioners...

    "provisioners": [
        {
            "type": "shell",
            "inline": [
                "rm -f /etc/yum.repos.d/*"
            ]
        },
        {
            "destination": "/etc/yum.repos.d/",
            "source": "repos/{{ user `repo_dir` }}/",
            "type": "file"
        },
        {
            "destination": "/etc/yum.repos.d/",
            "source": "repos/Common/",
            "type": "file"
        },

$ tree repos/
repos/
├── CentOS
│   └── CentOSArtifactory.repo
├── Common
│   └── EL-7-EPEL-x86_64.repo
└── Redhat
    ├── RHEL-7-Base-x86_64.repo
    └── RHEL-7-Common-x86_64.repo

NOTE: We use this same script to build RHEL7.x boxes, which is why we have the "repo_dir" variable (default to CentOS). All of the repo files just point to the internal Artifactory URL instead of the "normal" URLs (which we cannot reach). The "Common" directory used to have more repos in it, but now its just EPEL. We probably wouldn't even need EPEL at this stage anyhow, but its there.

Now on to the purpose, the reason we modified the repos in packer was to provide an "updated" image. We wanted to apply the updates in addition to just 7.5 or 7.6. I am not sure why they made it a script instead of just putting the commands directly into the json, but (shrug) :) ... this was added to the provisioners directly below the ones posted above:

        {
            "environment_vars": [
                "UPDATE=true"
            ],
            "scripts": [
                "scripts/updateos.sh"
            ],
            "type": "shell",
            "expect_disconnect": true,
            "pause_after": "20s"
        }

#! /bin/bash -x

if [[ $UPDATE  =~ true || $UPDATE =~ 1 || $UPDATE =~ yes ]]; then
 #   echo "==> Applying updates"
    yum -y update

# reboot
    echo "Rebooting the machine..."
    reboot
fi