Closed mancdaz closed 10 years ago
Comment by claco Wednesday Aug 20, 2014 at 17:33 GMT
Actually, it is in the repo: http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com/pool/main/d/dnsmasq/
, and it's dated the same as other packages.
Some things to check. When this has happened to me in the past, it's because the OS was deployed from a source/image that is newer than what we've created the frozen repo from. For example, a net-install that fetches the latest packages from the internet, rather than a specific set in an iso.
This means, if you're using a public cloud image, or a vagrant box, they could be newer, or are the very least, not the same as our kick images will be for real installs.
The safest thing atm is to start with an image that is 14.04.0, and dist-upgrade should do the right thing. That's not to say this isn't an actual issue. Just want to make sure we're starting from a correct base b4 applying the repo.
Comment by johnmarkschofield Wednesday Aug 20, 2014 at 17:36 GMT
This is using the image "14.04 LTS (Trusty Tahr) (PVHVM)" from mycloud.rackspace.com. I'm not sure if that's actually 14.04.1, but in any case, there's no other option on mycloud for a 14.04 release. (Other than the non-PVHVM image, which I'm assuming is the same base image.)
Comment by johnmarkschofield Wednesday Aug 20, 2014 at 17:36 GMT
I'm also going to build a box right now and verify this, just to make sure my scripts aren't doing something to upgrade prior to changing the sources.list.
Comment by johnmarkschofield Wednesday Aug 20, 2014 at 17:41 GMT
On a freshly-imaged "14.04 LTS (Trusty Tahr) (PVHVM)" server, on mycloud.rackspace.com, with no scripts or anything other than what you see here:
root@cloud-server-31:~# echo deb [arch=amd64] http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA main > /etc/apt/sources.list
root@cloud-server-31:~# cat /etc/apt/sources.list
deb [arch=amd64] http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA main
root@cloud-server-31:~# apt-get update
Get:1 http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA InRelease [1,704 B]
Ign http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA InRelease
Get:2 http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main amd64 Packages [2,116 kB]
Ign http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main Translation-en_US
Ign http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main Translation-en
Fetched 2,118 kB in 18s (114 kB/s)
Reading package lists... Done
W: GPG error: http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 443A026F1A3DF17E
root@cloud-server-31:~# curl http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com/repo.gpg | apt-key add -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3307 100 3307 0 0 5977 0 --:--:-- --:--:-- --:--:-- 5969
OK
root@cloud-server-31:~# apt-get update
Get:1 http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA InRelease [1,704 B]
Hit http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main amd64 Packages
Ign http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main Translation-en_US
Ign http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main Translation-en
Fetched 1,704 B in 12s (131 B/s)
Reading package lists... Done
root@cloud-server-31:~# apt-get install dnsmasq
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package dnsmasq is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
dnsmasq-base
E: Package 'dnsmasq' has no installation candidate
root@cloud-server-31:~#
Comment by johnmarkschofield Wednesday Aug 20, 2014 at 17:49 GMT
Reproduced on vagrant Ubuntu 14.04.0:
schof@bigbertha(5458):~/code/rackspace/dnsmasq_test:
$ vagrant init ubuntu/trusty64
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
schof@bigbertha(5459):~/code/rackspace/dnsmasq_test:
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ubuntu/trusty64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'ubuntu/trusty64' is up to date...
==> default: Setting the name of the VM: dnsmasq_test_default_1408556760173_92871
==> default: Clearing any previously set forwarded ports...
==> default: Fixed port collision for 22 => 2222. Now on port 2201.
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
==> default: Forwarding ports...
default: 22 => 2201 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2201
default: SSH username: vagrant
default: SSH auth method: private key
default: Warning: Connection timeout. Retrying...
default: Warning: Remote connection disconnect. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
default: /vagrant => /Users/schof/code/rackspace/dnsmasq_test
schof@bigbertha(5460):~/code/rackspace/dnsmasq_test:
$ vagrant ssh
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-29-generic x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Wed Aug 20 17:46:17 UTC 2014
System load: 0.15 Processes: 88
Usage of /: 2.7% of 39.34GB Users logged in: 0
Memory usage: 17% IP address for eth0: 10.0.2.15
Swap usage: 0%
Graph this data and manage this system at:
https://landscape.canonical.com/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
0 packages can be updated.
0 updates are security updates.
vagrant@vagrant-ubuntu-trusty-64:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04 LTS
Release: 14.04
Codename: trusty
vagrant@vagrant-ubuntu-trusty-64:~$ sudo su
root@vagrant-ubuntu-trusty-64:/home/vagrant# echo deb [arch=amd64] http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA main > /etc/apt/sources.list
root@vagrant-ubuntu-trusty-64:/home/vagrant# curl http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com/repo.gpg | apt-key add -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3307 100 3307 0 0 591 0 0:00:05 0:00:05 --:--:-- 811
OK
root@vagrant-ubuntu-trusty-64:/home/vagrant# apt-get update
Get:1 http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA InRelease [1,704 B]
Get:2 http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main amd64 Packages [2,116 kB]
Ign http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main Translation-en_US
Ign http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com LA/main Translation-en
Fetched 2,118 kB in 14s (147 kB/s)
Reading package lists... Done
root@vagrant-ubuntu-trusty-64:/home/vagrant# apt-get install dnsmasq
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package dnsmasq is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
dnsmasq-base
E: Package 'dnsmasq' has no installation candidate
root@vagrant-ubuntu-trusty-64:/home/vagrant#
Comment by johnmarkschofield Thursday Aug 21, 2014 at 18:52 GMT
@jacobwagner @hughsaunders Can either of you take a look at this? Or is there someone else I should ping?
Comment by aedan Friday Aug 22, 2014 at 13:47 GMT
I checked the offical trusty mirror and did not find dnsmasq either. It is apparently now called dnsmasq-base. The package dnsmasq-base is in the frozen repo. I attempted to install dnsmasq-base. After install I do have the dnsmasq daemon installed.
root@trusty-frozen:~# dnsmasq --help Usage: dnsmasq [options]
Valid options are:
-a, --listen-address=
I believe we just need to update the playbook to reference dnsmasq-base instead of dnsmasq
Comment by johnmarkschofield Friday Aug 22, 2014 at 17:49 GMT
@aedan Not sure about that. I can tell you with 100% certainty that you can install the dnsmasq package when using the Ubuntu standard repos, or our playbooks would never have worked.
I'll investigate why, and I'll test your dnsmasq-base fix.
Comment by aedan Friday Aug 22, 2014 at 17:56 GMT
@johnmarkschofield I checked the package list for the mirror.rackspace.com/ubuntu trusty repo and found:
Package: dnsmasq-base Priority: optional Section: net Installed-Size: 648 Maintainer: Ubuntu Developers ubuntu-devel-discuss@lists.ubuntu.com Original-Maintainer: Simon Kelley simon@thekelleys.org.uk Architecture: amd64 Source: dnsmasq Version: 2.68-1 Replaces: dnsmasq (<< 2.63-1~) Depends: adduser, libc6 (>= 2.15), libdbus-1-3 (>= 1.1.1), libidn11 (>= 1.13), libnetfilter-conntrack3 Breaks: dnsmasq (<< 2.63-1~) Filename: pool/main/d/dnsmasq/dnsmasq-base_2.68-1_amd64.deb Size: 257512 MD5sum: 60bafb9b863671bb02595505a447270c SHA1: ab4cba860fc099fd1badb07a68afddfd4b10bb5d SHA256: 8064b65d22724a4dca2a0cdbab8305b6a6054fac8e2b593dc1c1c2a11ff3ca24 Description: Small caching DNS proxy and DHCP/TFTP server Description-md5: 1f9c3f0c557ca377bcc6c659e4694437 Bugs: https://bugs.launchpad.net/ubuntu/+filebug Origin: Ubuntu Supported: 5y Task: virt-host, ubuntu-desktop, ubuntu-usb, kubuntu-desktop, kubuntu-full, kubuntu-active, kubuntu-active-desktop, kubuntu-active-full, kubuntu-active, edubuntu-desktop, edubuntu-usb, xubuntu-desktop, mythbuntu-frontend, mythbuntu-desktop, mythbuntu-backend-slave, mythbuntu-backend-master, lubuntu-desktop, ubuntustudio-desktop, ubuntu-gnome-desktop
So not that I don't believe you or anything. It just looks like it is being replaced by dnsmasq-base
Comment by johnmarkschofield Friday Aug 22, 2014 at 20:05 GMT
mirror.rackspace.com and archive.ubuntu.com have the same dnsmasq files:
$ curl -s http://mirror.rackspace.com/ubuntu/pool/main/d/dnsmasq/ | grep ".deb" | grep "href" | sed 's/^.*a href="\([-_.a-z0-9]*\).*$/\1/g' | sort
dnsmasq-base_2.52-1_amd64.deb
dnsmasq-base_2.52-1_i386.deb
dnsmasq-base_2.52-1ubuntu0.1_amd64.deb
dnsmasq-base_2.52-1ubuntu0.1_i386.deb
dnsmasq-base_2.59-4_amd64.deb
dnsmasq-base_2.59-4_i386.deb
dnsmasq-base_2.59-4ubuntu0.1_amd64.deb
dnsmasq-base_2.59-4ubuntu0.1_i386.deb
dnsmasq-base_2.66-4ubuntu1_amd64.deb
dnsmasq-base_2.66-4ubuntu1_i386.deb
dnsmasq-base_2.68-1_amd64.deb
dnsmasq-base_2.68-1_i386.deb
dnsmasq-base_2.71-1_amd64.deb
dnsmasq-base_2.71-1_i386.deb
dnsmasq-utils_2.59-4_amd64.deb
dnsmasq-utils_2.59-4_i386.deb
dnsmasq-utils_2.59-4ubuntu0.1_amd64.deb
dnsmasq-utils_2.59-4ubuntu0.1_i386.deb
dnsmasq-utils_2.66-4ubuntu1_amd64.deb
dnsmasq-utils_2.66-4ubuntu1_i386.deb
dnsmasq-utils_2.68-1_amd64.deb
dnsmasq-utils_2.68-1_i386.deb
dnsmasq-utils_2.71-1_amd64.deb
dnsmasq-utils_2.71-1_i386.deb
$ curl -s http://archive.ubuntu.com/ubuntu/pool/main/d/dnsmasq/ | grep ".deb" | grep "href" | sed 's/^.*a href="\([-_.a-z0-9]*\).*$/\1/g' | sort
dnsmasq-base_2.52-1_amd64.deb
dnsmasq-base_2.52-1_i386.deb
dnsmasq-base_2.52-1ubuntu0.1_amd64.deb
dnsmasq-base_2.52-1ubuntu0.1_i386.deb
dnsmasq-base_2.59-4_amd64.deb
dnsmasq-base_2.59-4_i386.deb
dnsmasq-base_2.59-4ubuntu0.1_amd64.deb
dnsmasq-base_2.59-4ubuntu0.1_i386.deb
dnsmasq-base_2.66-4ubuntu1_amd64.deb
dnsmasq-base_2.66-4ubuntu1_i386.deb
dnsmasq-base_2.68-1_amd64.deb
dnsmasq-base_2.68-1_i386.deb
dnsmasq-base_2.71-1_amd64.deb
dnsmasq-base_2.71-1_i386.deb
dnsmasq-utils_2.59-4_amd64.deb
dnsmasq-utils_2.59-4_i386.deb
dnsmasq-utils_2.59-4ubuntu0.1_amd64.deb
dnsmasq-utils_2.59-4ubuntu0.1_i386.deb
dnsmasq-utils_2.66-4ubuntu1_amd64.deb
dnsmasq-utils_2.66-4ubuntu1_i386.deb
dnsmasq-utils_2.68-1_amd64.deb
dnsmasq-utils_2.68-1_i386.deb
dnsmasq-utils_2.71-1_amd64.deb
dnsmasq-utils_2.71-1_i386.deb
Verification that they have identical versions:
$ curl -s http://mirror.rackspace.com/ubuntu/pool/main/d/dnsmasq/ | grep ".deb" | grep "href" | sed 's/^.*a href="\([-_.a-z0-9]*\).*$/\1/g' | sort | md5sum
5a26ff62fd0158fd4623650046cb3746 -
root@vagrant-ubuntu-trusty-64:/home/vagrant# curl -s http://archive.ubuntu.com/ubuntu/pool/main/d/dnsmasq/ | grep ".deb" | grep "href" | sed 's/^.*a href="\([-_.a-z0-9]*\).*$/\1/g' | sort | md5sum
5a26ff62fd0158fd4623650046cb3746 -
The pinned apt repo is missing most of these files:
$ curl -s http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com/pool/main/d/dnsmasq/ | grep ".deb" | grep "href" | sed 's/^.*a href="\([-_.a-z0-9]*\).*$/\1/g' | sort
dnsmasq-base_2.68-1_amd64.deb
dnsmasq-utils_2.68-1_amd64.deb
I looked at the main/d directory to count the number of packages:
$ curl -s http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com/pool/main/d/ | grep href | wc -l
108
$ curl -s http://archive.ubuntu.com/ubuntu/pool/main/d/ | grep href | wc -l
184
We're short 76 packages in our pinned repo. (Many or all of these may be unnecessary for our install; this isn't a conclusive test.)
I'm happy to do the updating on our pinned repo myself, but I don't know how to do that.
Comment by aedan Friday Aug 22, 2014 at 20:20 GMT
I would assume that it is a lot more than 76 packages missing. We only took the 64 bit packages and only the latest one at creation time. So it looks like in the case of dnsmasq there is now a 2.71 version. The goal was not to have every package, but to have the ones needed to fully deploy. If you run across any packages that are missing. Like this morning. I will add them or figure out what needs to be done to correct. It is not that we cannot have a full repo. I was just trying to reduce the size as much as possible. I believe the repo is currently 11GB in size where is could have been 68GB
Jake
Comment by johnmarkschofield Friday Aug 22, 2014 at 20:28 GMT
@aedan Harumph. I missed the obvious.
The reason we can install dnsmasq using the standard repos and not using our pinned repo is because the dnsmasq package is present in universe in the standard repos, and is not present in our merged & pinned repo. Here's the missing files:
http://archive.ubuntu.com/ubuntu/pool/universe/d/dnsmasq/
It's the 2.68-1 package we need.
Comment by jcannava Friday Aug 22, 2014 at 20:32 GMT
Out of curiosity, space aside, wouldn't it be easier to have everything from a maintenance perspective? That way we at least start from an everything point and only move forward what makes sense.
I really don't see the point of doing it this way, especially when we are asking people to lock with this.
Comment by aedan Friday Aug 22, 2014 at 20:34 GMT
No. We can do whatever. So I am thinking you would prefer then that I pull all packages? From universe and whatnot as well?
Comment by jcannava Friday Aug 22, 2014 at 20:39 GMT
I think it will save us a headache later, but that's my opinion curious what everyone else thinks.
Comment by johnmarkschofield Friday Aug 22, 2014 at 20:44 GMT
I don't think we need all packages -- we can omit the 32-bit ones, for instance.
In my experience if we pick and choose we're going to have a lot of cases just like this, so grabbing more-or-less wholesale is better.
In the past I've mirrored the repos and then used tags to manage them -- so we have a stable, alpha, RC, etc. repo, with each repo matching our git release tags. Not sure if the current infrastructure supports that.
@aedan Where is the code that generates the repo?
Comment by aedan Friday Aug 22, 2014 at 20:47 GMT
Currently the repo is maintained by hand using aptly on a cloud server in DFW called trusty-frozen. The plan is to automate it with the jenkins-rpc repo.
Comment by hughsaunders Tuesday Aug 26, 2014 at 12:13 GMT
@jcannava when you say 'everything' do you mean all of main, or main + universe? I'm guessing we can leave multiverse to prevent legal issues.
Comment by aedan Tuesday Aug 26, 2014 at 17:03 GMT
Ok its done.
deb [arch=amd64] http://dc0e2a2ef0676c3453b1-31bb9324d3aeab0d08fa434012c1e64d.r5.cf1.rackcdn.com universe-LA universe
This issue should be fixed. The dnsmasq package is available in LA-universe
Issue by johnmarkschofield Wednesday Aug 20, 2014 at 16:36 GMT Originally opened as https://github.com/rcbops/ansible-lxc-rpc-orig/issues/426
When using the pinned apt repo:
The dnsmasq package is not available:
This causes the openstack-common.yml playbook to fail:
This is a release-blocker.