Experiment 03_compute_nodes_unavailable not working

[snrlx]

The experiment stops as it is waiting for an instance to be started with the following command line output:

+ Compute Node Availability
  * SETUP
    - create test instance while everything is working fine

    - wait for instance to come up

When checking Horizon it is visible that the instance is actually up and running so we need to find out what the actual problem is.

[snrlx]

The experiment hangs in that loop:

subheading "wait for instance to come up"
until [ "`ssh-keyscan -H $FLOATING_IP1 2> /dev/null`" != "" ]
do
    sleep 1
    log -vvv -n .
done

[snrlx]

In this line they grep for a floating ip: https://github.com/MasterprojectOpenStack1516/sourcecode/blob/master/experiments/common/create_openstack_instance_named#L19

As they use

FLOATING_IP=`tail -n1 "$TMP_PATH/remote.log" | 
grep "FLOATING_IP:"| cut -s -d ":" -f2`

they only check for the last line. In my case the log looked like this:

ERROR (BadRequest): No nw_info cache associated with instance (HTTP 400) (Request-ID: req-2cfe7ec7-2487-4fdb-a139-23a645312132)

Apparently the instance doesn't get a floating IP assigned, which of course is problematic.

Problem seems to be apparent to other users too: https://bugs.launchpad.net/nova/+bug/1407417

[snrlx]

After having multiple temporary errors that were connected to an inconsistent network cache and authentication issues I now saw the experiment continue. It simply takes 5 minutes until the experiment notices the instance as started because after that time the ssh keys are present and therefore the experiment continues. I suggest writing a simple note into the experiment that this step might take up some time!

[snrlx]

Connected to pull request https://github.com/MasterprojectOpenStack1516/sourcecode/pull/27