Integration test infrastructure

[kotfic]

Still planning on writing some code that wraps the command and allows us to do fully automated testing. To check out just the command you can run:

    cumulus full_status > before
    cumulus -v create_profile \
           create_cluster \
           launch_cluster \
           create_volume \
           attach_volume

    cumulus full_status > during

    cumulus -v detach_volume \
           delete_volume \
           terminate_cluster \
           delete_cluster \
           delete_profile

    cumulus full_status > after

Then, to verify the state you should then be able to run something like:

sdiff -w 300 before during
sdiff -w 300 during after
sdiff -w 300 before after

[kotfic]

A few things to change after meeting with @cjh1

• [x] Reorganizing wait_for_status into check_log because of complexity linting has introduced a bug. Not all ansible log messages are displaying (task still completes correctly).
• [x] CLI tool should chain commands in a more natural way (e.g. instead of cumulus create_cluster it should be cumulus cluster create, cumulus cluster launch etc)
• [x] CLI tools should not create resources just to delete them (e.g. cumulus delete_cluster this is fine for testing)
• [x] Test infrastructure should warn (instead of fail) if profiles/clusters/etc exist in the database
• [x] Add click documentation to subcommands
• [x] Add documentation for CLI command
• [x] Add documentation for integration testing command

[kotfic]

@cjh1 This is ready for review. You may want to start with documentation for the command and the integration tests

I have not tested that all the integration tests are passing (e.g. test_traditional). As long as the ansible related tests are working for you, i'd like to fix those in another PR (one that also removed the 'legacy' integration scripts in tests/integration/).

[cjh1]

@kotfic Thanks for pushing this forward, this is going to be very useful. I will review soon.

[cjh1]

@kotfic I am starting to look at this, I think it currently making the assumption that all cluster are cloud clusters. In my current environment I have several traditional clusters. If I issue:

cumulus list_clusters

I get:

Traceback (most recent call last):
  File "/home/cjh/virtualenv/cumulus/bin/cumulus", line 9, in <module>
    load_entry_point('cumulus==0.9.0', 'console_scripts', 'cumulus')()
  File "/home/cjh/virtualenv/cumulus/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/home/cjh/virtualenv/cumulus/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/home/cjh/virtualenv/cumulus/local/lib/python2.7/site-packages/click/core.py", line 1086, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/home/cjh/virtualenv/cumulus/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/cjh/virtualenv/cumulus/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/home/cjh/virtualenv/cumulus/local/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "/home/cjh/virtualenv/cumulus/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/home/cjh/work/source/cumulus/cumulus/scripts/command.py", line 81, in list_clusters
    print tabulate([[f(c) for f in keys] for c in proxy.clusters],
  File "/home/cjh/work/source/cumulus/cumulus/scripts/utils.py", line 62, in _profile
    return profile_dict[instance['profileId']]
KeyError: 'profileId'

[cjh1]

@kotfic Here is some JSON to create a trad cluster:

{
  "config": {
    "ssh": {
      "user": "billy"
    },
    "host": "hell"
  },
    "name": "TradIntegrationTest",
    "type": "trad"
}

[kotfic]

@cjh1 this should fix the traditional cluster issue. thanks!