Integration tests

[awestendorf]

Implements a simple framework for writing integration tests. The purpose is to test features and suites of behaviors against a real AMQP host. This should make it easier to write tests for complex features and bug fixes.

Right now, the "framework" is just a shell script that looks for executable test files in the integration directory. In the future, boilerplate may be extracted away so that users have control over the target host, virtual host, passwords and so on.

These tests live outside of the tests directory as they can't be run on Travis CI. They can be run from the root project directory by calling ./scripts/integration

[awestendorf]

@vitaly-krugl I'd like this or something like it as a basis for better tests on #80 and #82.

[vitaly-krugl]

@awestendorf - it's great to have integration tests in the project! I have some feedback:

1. It would be awesome to get integration tests running on Travis CI (pika does this, for example). What is your reasoning for not running integration tests on Travis CI?
2. I would prefer a standard test framework. The asserts in test-get.py won't display useful info when the assertions fail. On the other hand, self.assertEqual, etc., in unittest (e.g.) automatically output useful information about the failure that helps tremendously with debugging. Builtin fixtures, such as addCleanup, setUp, and tearDown, are extremely useful.
3. Although I've somewhat adopted to Chai when working on Haigha unit tests, unittest would be my personal preference, because more developers are proficient with it, making it easier and faster to write quality tests.
4. On occasion, I find it useful to use mock in integration tests as well: e.g., mocking out a timeout duration constant with a much shorter value to run the integration test in practical amount of time.
5. I like py.test as a test runner. It has good options, such as "boxing" tests inside individual processes to prevent unintended side-effects from one test affecting other tests.

[vitaly-krugl]

Hi Aaron, I left some feedback regarding integration tests in your PR https://github.com/agoragames/haigha/pull/85. I will add an integration test for my PR #82 (server-initiated basic.cancel).

Also, my PR #80 (https://github.com/agoragames/haigha/pull/80) was meant only as an example for Kevin Conway's PR #69. I just closed #80. I plan to review Kevin's PR #69 this week.

Best, Vitaly

From: Aaron Westendorf notifications@github.com<mailto:notifications@github.com> Reply-To: agoragames/haigha reply@reply.github.com<mailto:reply@reply.github.com> Date: Monday, July 6, 2015 at 7:25 AM To: agoragames/haigha haigha@noreply.github.com<mailto:haigha@noreply.github.com> Cc: Vitaly Kruglikov vkruglikov@numenta.com<mailto:vkruglikov@numenta.com> Subject: Re: [haigha] Integration tests (#85)

@vitaly-kruglhttps://github.com/vitaly-krugl I'd like this or something like it as a basis for better tests on #80https://github.com/agoragames/haigha/pull/80 and #82https://github.com/agoragames/haigha/pull/82.

Reply to this email directly or view it on GitHubhttps://github.com/agoragames/haigha/pull/85#issuecomment-118871664.

[awestendorf]

I didn't think Travis supported Rabbit, but knowing that, tests/integration is the right place to put the tests. I think the rest of your comments flow from there. Chai is a superset of unittest, you should be able to continue using all of its features.

I'm going to close this and open another one with new integration tests.

[vitaly-krugl]

Aaron, please see this for simple instructions on starting RabbitMQ on Travis-CI: http://docs.travis-ci.com/user/database-setup/#RabbitMQ

[vitaly-krugl]

Aaron, I enabled rabbitmq service in travis via PR #82, which has integration tests. See comments in #82.