j2labs / brubeck

Asynchronous web and messaging
http://brubeck.io
511 stars 66 forks source link

Tell Redis to do flushdb in TestRedisQueryset.setUp so that tests always... #92

Closed msabramo closed 12 years ago

msabramo commented 12 years ago

... start with a clean slate.

Without this, I get the following error on most test runs, because Redis will persist the state from previous runs.

marc@hyperion:~/dev/git-repos/brubeck
13:29:27 $ tox -e py27
_________________________________________________________________________________ [tox sdist] __________________________________________________________________________________
[TOX] ***creating sdist package
[TOX] /Users/marc/dev/git-repos/brubeck$ /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
[TOX] ***copying new sdistfile to '/Users/marc/.tox/distshare/brubeck-0.4.0.zip'
______________________________________________________________________________ [tox testenv:py27] ______________________________________________________________________________
[TOX] ***reusing existing matching virtualenv py27
[TOX] ***upgrade-installing sdist
[TOX] /Users/marc/dev/git-repos/brubeck/.tox/py27/log$ ../bin/pip install --download-cache=/Users/marc/dev/git-repos/brubeck/.tox/_download -U --no-deps ../../dist/brubeck-0.4.0.zip >22.log
[TOX] /Users/marc/dev/git-repos/brubeck$ .tox/py27/bin/nosetests --exe -w tests
.........F.........................
======================================================================
FAIL: test__create_many (tests.test_queryset.TestRedisQueryset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/marc/dev/git-repos/brubeck/tests/test_queryset.py", line 229, in test__create_many
    self.assertEqual(self.queryset.MSG_CREATED, status)
AssertionError: 'Created' != 'Updated'

----------------------------------------------------------------------
Ran 35 tests in 0.064s

FAILED (failures=1)
[TOX] ERROR: InvocationError: '.tox/py27/bin/nosetests --exe -w tests'
________________________________________________________________________________ [tox summary] _________________________________________________________________________________
[TOX] ERROR: py27: commands failed
j2labs commented 12 years ago

Good call.

stuntgoat commented 12 years ago

Nice catch @msabramo !

msabramo commented 12 years ago

There is another potential problem here. In order to run the tests, I have to run a copy of redis-server, which isn't too bad, but what if someone else is already running a redis-server with real data in it? And then they run these tests, then won't the flushdb wipe out their data?

Perhaps a better approach is to mock out the Redis connection, perhaps using something like https://github.com/locationlabs/mockredis ?

gone commented 12 years ago

I'm not super knowledgeable about redis - is it possible to force an unused db to be used?

On Mon, Nov 12, 2012 at 11:37 AM, Marc Abramowitz notifications@github.comwrote:

There is another potential problem here. In order to run the tests, I have to run a copy of redis-server, which isn't too bad, but what if someone else is already running a redis-server with real data in it? And then they run these tests, then won't the flushdb wipe out their data?

Perhaps a better approach is to mock out the Redis connection, perhaps using something like https://github.com/locationlabs/mockredis ?

— Reply to this email directly or view it on GitHubhttps://github.com/j2labs/brubeck/pull/92#issuecomment-10294378.

sethmurphy commented 12 years ago

As far as flushing production data or important data you should not be running tests on a production box in hte first place.

However, there may be important dev data from another project. In order to minimize the chance of flushing an existing DB, what about simply running Redis on a non standard port for the tests?

On Mon, Nov 12, 2012 at 11:37 AM, Marc Abramowitz notifications@github.comwrote:

There is another potential problem here. In order to run the tests, I have to run a copy of redis-server, which isn't too bad, but what if someone else is already running a redis-server with real data in it? And then they run these tests, then won't the flushdb wipe out their data?

Perhaps a better approach is to mock out the Redis connection, perhaps using something like https://github.com/locationlabs/mockredis ?

— Reply to this email directly or view it on GitHubhttps://github.com/j2labs/brubeck/pull/92#issuecomment-10294378.

gone commented 12 years ago

Although it's true that tests shouldn't be run on prod, it would only take a single person to do that to destroy our credibility.

On Mon, Nov 12, 2012 at 11:46 AM, Seth Murphy notifications@github.comwrote:

As far as flushing production data or important data you should not be running tests on a production box in hte first place.

However, there may be important dev data from another project. In order to minimize the chance of flushing an existing DB, what about simply running Redis on a non standard port for the tests?

On Mon, Nov 12, 2012 at 11:37 AM, Marc Abramowitz notifications@github.comwrote:

There is another potential problem here. In order to run the tests, I have to run a copy of redis-server, which isn't too bad, but what if someone else is already running a redis-server with real data in it? And then they run these tests, then won't the flushdb wipe out their data?

Perhaps a better approach is to mock out the Redis connection, perhaps using something like https://github.com/locationlabs/mockredis ?

— Reply to this email directly or view it on GitHub< https://github.com/j2labs/brubeck/pull/92#issuecomment-10294378>.

— Reply to this email directly or view it on GitHubhttps://github.com/j2labs/brubeck/pull/92#issuecomment-10294714.

msabramo commented 12 years ago

https://github.com/locationlabs/mockredis looks to be missing functionality that brubeck needs such as the hvals and reset commands and its version of pipeline execute doesn't return a list of values.

What appears to be a better fit is https://github.com/jamesls/fakeredis

msabramo commented 12 years ago

Another option is to create a redis.conf that launches a Redis server in the Brubeck tests directory using a UNIX domain socket. Ideally the tests could even start and stop such a server. Some ideas at https://github.com/mranney/node_redis/issues/204

msabramo commented 12 years ago

See https://github.com/j2labs/brubeck/pull/94 for a PR to replace the use of real Redis with fakeredis.