Add end to end large tests with plugin tasks

[nanliu]

The ethtool plugin requires specific hardware to work correctly. For now this is not possible to specify in docker so this set of end to end tests use vagrant and e1000 nic to verify the plugin works as expected.

[nanliu]

Here's the example output:

==> centos72: Loaded plugins: fastestmirror
==> centos72: Determining fastest mirrors
==> centos72:  * base: mirrors.gigenet.com
==> centos72:  * extras: centos.mirrors.wvstateu.edu
==> centos72:  * updates: mirror.trouble-free.net
==> centos72: Package 2:ethtool-3.15-2.el7.x86_64 already installed and latest version
==> centos72: Nothing to do
==> centos72: snap provision complete
==> centos72: Running provisioner: serverspec...

Command "/opt/snap/bin/snapctl plugin load /opt/snap/plugins/snap-plugin-publisher-file"
  exit_status
.    should eq 0

Command "/opt/snap/bin/snapctl plugin load /opt/snap/plugins/snap-plugin-collector-ethtool"
  exit_status
.    should eq 0

Command "/opt/snap/bin/snapctl plugin list"
  exit_status
.    should eq 0
  stdout
.    should match /file/
  stdout
.    should match /ethtool/

Command "/opt/snap/bin/snapctl task create -t /vagrant/examples/tasks/ethtool-file.json"
  exit_status
.    should eq 0

Command "sleep 30"
  exit_status
.    should eq 0

File "/tmp/published_netstats"
.  should be file
  content
.    should match /rx_bytes/

Finished in 34.61 seconds (files took 0.05365 seconds to load)
9 examples, 0 failures

[candysmurf]

👍 , LGTM

[candysmurf]

@nanliu, How will these tests run with containers, is this something you'll look into next?

[nanliu]

This is a PoC, we should have a discussion on what tools/framework we want to use for large end to end tests. I'm going to leave this open until we decide the best way to move forward.

[kindermoumoute]

@nanliu, the way you are doing it with Vagrant is not that different from using containers. It could totally work with this proposal, docker-compose.yml would become Vagrantfile, and example.shwould be example.rb.

[nanliu]

The goal is not to use vagrant (that's a one off due to this plugin), but use docker/docker-compose. This test is kicked off by running vagrant up and does not need an example.rb. For docker, the spec tests will remain the same and I think the spec_helper would do some detection and switch to the docker backend. I should write another docker example to compliment this one.

[nanliu]

We will merge this for now since there's no other way to test this plugin easily.