Add Fedora 21 Beaker node

[dfarrell07]

Fedora 21 is a critical OS to test against, just below CentOS 7. Unfortunately, the Chef folks haven't released a Vagrant base box for it yet. I raised an issue to get the process started, but I have a good bit of learning to do before I could do much to contribute to the creation effort. This TODO might take a while to complete.

A short-term, fairly undesirable fix is to use one of the very-unofficial Fedora 21 boxes on Vagrantcloud. I'm not thrilled about this for all the reasons, but it's better than not testing Fedora 21 at all.

[dfarrell07]

Trying to use the F21 box linked above, I'm seeing the following error:

Exited: 1
/home/daniel/.gem/ruby/gems/beaker-2.2.0/lib/beaker/host.rb:285:in `exec': Host 'fedora-21-x64' exited with 1 running: (Beaker::Host::CommandFailure)
 yum install -y puppet
Last 10 lines of output were:
    --> Running transaction check
    ---> Package libyaml.x86_64 0:0.1.6-6.fc21 will be installed
    ---> Package ruby-irb.noarch 0:2.1.5-26.fc21 will be installed
    ---> Package ruby-shadow.x86_64 1:2.2.0-2.el7 will be installed
    --> Processing Dependency: libruby.so.2.0()(64bit) for package: 1:ruby-shadow-2.2.0-2.el7.x86_64
    --> Finished Dependency Resolution
    Error: Package: 1:ruby-shadow-2.2.0-2.el7.x86_64 (puppetlabs-deps)
               Requires: libruby.so.2.0()(64bit)
     You could try using --skip-broken to work around the problem
     You could try running: rpm -Va --nofiles --nodigest

With this config file:

HOSTS:
  fedora-21-x64:
    roles:
      - master
    platform: el-7-x86_64
    box: TFDuesing/Fedora-21
    hypervisor: vagrant

CONFIG:
  log_level: verbose
  type: foss

[dfarrell07]

Narrowed down the scope of the error described above.

The Beaker config above produced the following Vagrantfile:

Vagrant.configure("2") do |c|
  c.vm.define 'fedora-21' do |v|
    v.vm.hostname = 'fedora-21'
    v.vm.box = 'TFDuesing/Fedora-21'
    v.vm.box_url = 'https://vagrantcloud.com/TFDuesing/boxes/Fedora-21'
    v.vm.box_check_update = 'true'
    v.vm.network :private_network, ip: "10.255.118.215", :netmask => "255.255.0.0", :mac => "0800278DA5F2"
    v.vm.provider :virtualbox do |vb|
      vb.customize ['modifyvm', :id, '--memory', '1024']
    end
  end
end

Which I copied into a new directory, then vagrant up'd.

A sudo yum install -y puppet in the resulting box works as expected. However, parsing through the output produce by Beaker, I noticed that it doesn't install Puppet from the default repos. Instead, it uses rpm -q --quiet puppetlabs-release || rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm and then yum install -y puppet.

Repeating those two commands on a fresh box produced from the Vagrantfile above was enough to reproduce the error:

Error: Package: 1:ruby-shadow-2.2.0-2.el7.x86_64 (puppetlabs-deps)
           Requires: libruby.so.2.0()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

[dfarrell07]

Even cleaner, I can reproduce the error with the following.

Add the chef/fedora-21 Vagrant box described here.

wget http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_fedora-21_chef-provisionerless.box
vagrant box add chef/fedora-21 opscode_fedora-21_chef-provisionerless.box

Create a working directory and this Vagrantfile

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "chef/fedora-21"
  config.vm.provision "shell", inline: "rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm"
  config.vm.provision "shell", inline: "yum install -y puppet"
end

vagrant up

==> default: Error: Package: 1:ruby-shadow-2.2.0-2.el7.x86_64 (puppetlabs-deps)
==> default:            Requires: libruby.so.2.0()(64bit)
==> default:  You could try using --skip-broken to work around the problem
==> default:  You could try running: rpm -Va --nofiles --nodigest

[dfarrell07]

Reading the relevant Puppet docs, it seems plausible that this error is caused by Beaker attempting to use the CentOS/RHEL 7 Puppet repo, instead of the Fedora one. Note that there isn't a Fedora 21 RPM at the URL bumped from F20.

http://yum.puppetlabs.com/puppetlabs-release-fedora-21.noarch.rpm

Not Found

The requested URL /puppetlabs-release-fedora-21.noarch.rpm was not found on this server.

[dfarrell07]

Using bundle exec rake fedora to run against F20, I see Beaker issuing the same RPM command described above for F21.

fedora-20 13:25:21$ rpm -q --quiet puppetlabs-release || rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm

For whatever reason, this doesn't cause yum install -y puppet to fail.

[dfarrell07]

This Beaker node file

HOSTS:
  fedora-21:
    roles:
      - master
    platform: fedora-21-x86_64
    # TODO: Move to an offical-ish F21 box once one goes on Vagrantcloud
    # See: https://github.com/opscode/bento/issues/312
    box: TFDuesing/Fedora-21
    box_url: https://vagrantcloud.com/TFDuesing/boxes/Fedora-21
    hypervisor: vagrant

CONFIG:
  log_level: debug
  type: foss

Produces this error (due to missing F21 RPM).

/home/daniel/.gem/ruby/gems/beaker-2.2.0/lib/beaker/host.rb:285:in `exec': Host 'fedora-21' exited with 1 running: (Beaker::Host::CommandFailure)
 rpm -q --quiet puppetlabs-release || rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-fedora-21.noarch.rpm
Last 10 lines of output were:
    curl: (22) The requested URL returned error: 404 Not Found
    error: skipping http://yum.puppetlabs.com/puppetlabs-release-fedora-21.noarch.rpm - transfer failed
    Retrieving http://yum.puppetlabs.com/puppetlabs-release-fedora-21.noarch.rpm

Note that the 404 is expected, but it's good confirmation to see that the URL is determined by the platform line in the Beaker node description.

[dfarrell07]

Changing the platform line to point at F20, and therefore a working RPM URL, gets things mostly working for F21. I'm seeing one Beaker test failure (at least they are running).

Failures:

  1) opendaylight class default parameters Service "opendaylight" should be enabled
     On host ``
     Failure/Error: it { is_expected.to be_enabled }
       systemctl --plain list-dependencies runlevel3.target | grep '^opendaylight.service$'
       expected Service "opendaylight" to be enabled
     # ./spec/acceptance/class_spec.rb:22:in `block (4 levels) in <top (required)>'

Finished in 2 minutes 18.8 seconds
4 examples, 1 failure

Failed examples:

rspec ./spec/acceptance/class_spec.rb:22 # opendaylight class default parameters Service "opendaylight" should be enabled

[dfarrell07]

The issue is that as of Fedora 21, systemctl --plain list-dependencies outputs a green or red (stdout, stderr) unicode circle (9679) and a space before the service name. The grep command executed by serverspec matches against the beginning of the line, which was fine for F20 without the prepended unicode but fails for F21, even if the service is enabled.

[vagrant@localhost ~]$ systemctl --plain list-dependencies runlevel3.target | grep 'opendaylight.service$'
● opendaylight.service
[vagrant@localhost ~]$ echo $?
0
[vagrant@localhost ~]$ systemctl --plain list-dependencies runlevel3.target | grep '^opendaylight.service$'
[vagrant@localhost ~]$ echo $?
1

[dfarrell07]

These are the lines of systemctl that are causing the issue:

https://github.com/systemd/systemd/blob/master/src/systemctl/systemctl.c#L1629

[dfarrell07]

This is the commit that caused the issue:

https://github.com/systemd/systemd/commit/dbc2c080debdc45683bc5534e327455d4fbbbea7

[dfarrell07]

The systemd people on Freenode/#systemd made a quick fix to cause --plain to not output the unicode dots that are causing issues.

3:07:54 PM <dfarrell07> I'm interested in disabling the unicode char printed by `systemctl list-dependices`, added in this commit https://github.com/systemd/systemd/commit/dbc2c080debdc45683bc5534e327455d4fbbbea7
3:08:33 PM <dfarrell07> It's causing issues for projects that grep for `^<some service>$`
3:09:03 PM <dfarrell07> Because of the matching against the beginning of the line
3:09:18 PM <dfarrell07> A popular tool, serverspec, is hitting a bug for F21 because of this
3:09:33 PM — dfarrell07 is digging into that bug from a distantly related project
3:10:31 PM  → Noiwex joined  ⇐ normalrawr quit  
3:16:28 PM <zbyszek> dfarrell07: systemctl show -p Wants,Requires ... is the proper way to do this.
3:16:48 PM <zbyszek> That output is stable and should be used by other tools.
3:16:59 PM <dfarrell07> zbyszek: checking that out now, thanks for info
3:17:04 PM → The_Blob joined (~blob@S010600c00cb01b8c.vs.shawcable.net)
3:17:13 PM <zbyszek> systemctl "pretty" output is explicitly documented as user-oriented and subject ot change.
3:17:25 PM <dfarrell07> zbyszek: cool, makes sense
3:17:36 PM → bel3atar_ joined (~bel3atar@paranoia.virtual-dope.com)
3:17:45 PM <grawity> Wants, Requires are not transitive though
3:18:06 PM  → stackofcats joined  ⇐ bel3atar, manio, stackofc1ts, [42], mamikk, booly-yam-5635 and Kinokoio quit  
3:18:45 PM <zbyszek> grawity: True...
3:19:55 PM <dfarrell07> grawity: was just noticing that, as I can't duplicate the behavior serverspec seems to need using `show -p <whatever>`
3:21:36 PM → [42] joined (N4Y@nothing4you.w.tf-w.tf)
3:22:12 PM <zbyszek> I guess we could modify --plain to not show the dot.
3:22:29 PM <dfarrell07> zbyszek: that's what I initially had in mind
3:22:37 PM <dfarrell07> read: that's the behavior I expected
3:22:39 PM  → manio and prahal joined  
3:23:40 PM <dfarrell07> zbyszek, grawity: Would you like me to file a bug to document this?
3:23:50 PM <zbyszek> Not necessary.
3:25:11 PM ⇐ ||arifaX quit (~Commodore@unaffiliated/arifax/x-427475) Quit: http://quassel-irc.org - Chat comfortably. Anywhere.
3:27:11 PM <zdzichu> This function is intended to generate human-readable output. If you are looking for computer-parsable output, use show instead. 
3:27:18 PM <zdzichu> ^^ mam systemctl status
3:27:19 PM <zdzichu> man
3:27:53 PM → mamikk joined (~mamikk@130.170.202.84.customer.cdi.no)
3:29:39 PM <dfarrell07> zdzichu: Given that `show` seems to be non-transitive, how can I reproduce the behavior of commands like `systemctl --plain list-dependencies runlevel3.target | grep '^opendaylight.service$'` using it?
3:30:42 PM ⇐ xakz quit (~XaMaD@176.186.205.89) Ping timeout: 245 seconds
3:30:47 PM <zdzichu> ehm, what are you trying to do?
3:30:53 PM ⇐ Noiwex quit (~nx@109.188.124.53) Ping timeout: 240 seconds
3:31:03 PM <zdzichu> it looks like "systemctl is-enabled opendaylight.service"
3:31:15 PM <dfarrell07> zdzichu: not so much me as a popular project, called serverspec, that uses basically exactly that command
3:31:16 PM ⇐ kukks quit (~Guenter@samba/team/kukks) Ping timeout: 264 seconds
3:31:23 PM — dfarrell07 looks into `is-enabled`
3:32:48 PM <dfarrell07> zdzichu: is-enabled looks like what they need, I'll submit a PR to move them to that interface
3:32:51 PM <dfarrell07> thanks @ all :)
3:33:53 PM  → booboy and booly-yam-4014 joined  ⇐ bluelupo quit  ↔ stackofcats and manio nipped out  
3:42:17 PM <fdo-vcs> zbyszek master 250ba66 src/systemctl/systemctl.c * systemctl: do not show dots with --plain
3:42:49 PM <zbyszek> dfarrell07: ^, even if you don't need actually need it.
3:43:19 PM <dfarrell07> zbyszek: Awesome, thanks *impressed* :)

[dfarrell07]

Went over to the serverspec repo with the intention of filing a bug to move them to the is-enabled systemd interface. However, it seems that the new-ish versions of serverspec have already made the switch.

Looking at the commit that cut the 1.16.0 release, which is the one Bundler is giving me, I see the relevant (erroneous) lines.

[~/serverspec]$ git checkout -b systemd_f21 5a38362 
[~/serverspec]$ grep -rniI "list-d" *
spec/arch/service_spec.rb:7:  its(:command) { should eq "systemctl --plain list-dependencies multi-user.target | grep '^sshd.service$'" }
spec/arch/service_spec.rb:16:  its(:command) { should eq "systemctl --plain list-dependencies graphical.target | grep '^sshd.service$'" }
spec/nixos/service_spec.rb:7:  its(:command) { should eq "systemctl --plain list-dependencies multi-user.target | grep 'sshd.service$'" }
spec/nixos/service_spec.rb:16:  its(:command) { should eq "systemctl --plain list-dependencies multi-user.target | grep 'sshd.service$'" }
spec/nixos/service_spec.rb:21:  its(:command) { should eq "systemctl --plain list-dependencies graphical.target | grep 'sshd.service$'" }
spec/redhat7/service_spec.rb:7:  its(:command) { should eq "systemctl --plain list-dependencies runlevel3.target | grep '^sshd.service$'" }
spec/redhat7/service_spec.rb:16:  its(:command) { should eq "systemctl --plain list-dependencies runlevel4.target | grep '^sshd.service$'" }

Looking at the latest version, they are gone.

[~/serverspec]$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
[~/serverspec]$ grep -rniI "list-d" *
[~/serverspec]$

[dfarrell07]

Now I'm in a bit of a dependency hell.

I need a newer version of serverspec to get the systemd fix to make F21 work. However, when I try to request a newer version from Bunlder (Gemfile below) it helpfully informs me of the following conflict:

[~/dfarrell07-opendaylight]$ bundle install
Updating https://github.com/rodjek/rspec-puppet.git
Fetching gem metadata from https://rubygems.org/.........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Bundler could not find compatible versions for gem "rspec":
  In Gemfile:
    rspec-puppet (>= 0) ruby depends on
      rspec (~> 2.0) ruby

    serverspec (~> 2.0) ruby depends on
      rspec (3.0.0)

[~/dfarrell07-opendaylight]$ cat Gemfile                                                                                                               15:48:14
source "https://rubygems.org"

group :test do
  gem "rake"
  gem "puppet", ENV['PUPPET_VERSION'] || '~> 3.7.0'
  gem "rspec-puppet", :git => 'https://github.com/rodjek/rspec-puppet.git'
  gem "puppetlabs_spec_helper"
  gem "metadata-json-lint"
end

group :development do
  gem "travis"
  gem "serverspec", "~> 2.0"
  gem "travis-lint"
  gem "beaker"
  gem "beaker-rspec"
  gem "vagrant-wrapper"
  gem "puppet-blacksmith"
  gem "guard-rake"
end

[dfarrell07]

Found a pull request to update the version of rspec used by rspec-puppet. If this gets merged, my issues should be solved. I posted there to let them know about the issues with Beaker testing against F21 that dep issue is causing.

[dfarrell07]

Someone forked rspec-puppet and only updated the version of rspec required. Updating my Gemfile to point at that fork, deleting Gemfile.lock and re-running bundle install gave me a version of serverspec with the F21 systemd fix I need.

[~/dfarrell07-opendaylight]$ bundle exec gem list | grep rspec                                                                                         18:21:05
beaker-rspec (5.0.0)
rspec (3.1.0)
rspec-core (3.1.7)
rspec-expectations (3.1.2)
rspec-its (1.1.0)
rspec-mocks (3.1.3)
rspec-puppet (2.0.0)
rspec-support (3.1.2)
serverspec (2.7.1)

[dfarrell07]

And finally, Beaker against Fedora 21 passes.

Finished in 2 minutes 21.7 seconds (files took 2 minutes 48.8 seconds to load)
4 examples, 0 failures

[dfarrell07]

The bulk of this work is done. Going to handle the remaining, more specific TODOs in their own issues. For now, those are #15 and #16.