search

FinalsClub / karmaworld

KarmaNotes.org v3.0
GNU Affero General Public License v3.0
7 stars 6 forks source link

testCourseUniqueness errors from transaction rollback #278

Closed btbonval closed 10 years ago

btbonval commented 10 years ago

After the expected IntegrityError occurs, a count() is performed, but appears to trip up due to the outstanding Transaction.

Traceback (most recent call last):
  File "/home/vagrant/karmaworld/karmaworld/apps/courses/test/test.py", line 26, in testCourseUniqueness
    self.assertEqual(Course.objects.count(), 1)
...
  File "/var/www/karmaworld/venv/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 54, in execute
    return self.cursor.execute(query, args)
DatabaseError: current transaction is aborted, commands ignored until end of transaction block

This seemed to have worked in Django 1.4 but breaks in Django 1.5. The online doc style has been changed so it is hard to compare, but I didn't see anything in the changelog overview re: testing.

charlesconnell commented 10 years ago

Dealt with in 836f891. Change in behavior probably has to do with the "Note" here https://docs.djangoproject.com/en/1.5/topics/testing/overview/#django.test.TransactionTestCase

btbonval commented 10 years ago

I'm not sure that note is the problem, because the change was that transactions were reset either before or after the test was run. I'm fairly sure the count() error is failing due to the IntegrityError. A reset before or after the test won't make a difference, as both calls are inside the test.

We need to tell Django to end the transaction and start a new one after the IntegrityError, but I'm not finding out how to do that.

btbonval commented 10 years ago

Can we do something using the same sort of with(transaction) as in commit e75eb9861010bc7a5eaed01712cbdaaec4acfdd8 to properly test nothing was added?

JHilker commented 10 years ago

Looked into this and uncommented the count line. It consistently passes without error on the current master. Could something we have done in the past two weeks accidentally remedied the problem?

charlesconnell commented 10 years ago

@JHilker You're using sqlite right? Try postgres.

btbonval commented 10 years ago

I thought Django had some kind of pseudo-rollback that worked with the sqlite backend? I'm not citing any sources, so I might very well have made that up in my memory.

charlesconnell commented 10 years ago

Removed problematic line in. 47c7e496adb63450e3f7372d82d9b7751020b4d1. Spent half an hour trying to make this work, wasn't worth it. It's not a big deal.

btbonval commented 10 years ago

Yeah, I think that's okay. I'm personally more worried about integration testing that ensures the actions users can take work as expected. This sort of test seems more about ensuring Django and psycopg are working than ensuring our code works. Maybe I'm the only one who sees it that way.

AndrewMagliozzi commented 10 years ago

I'd love to implement some integration testing. @WilliamBratches are you interested in taking on that task starting next week?

On Wed, Feb 26, 2014 at 12:18 AM, Bryan Bonvallet notifications@github.comwrote:

Yeah, I think that's okay. I'm personally more worried about integration testing that ensures the actions users can take work as expected. This sort of test seems more about ensuring Django and psycopg are working than ensuring our code works. Maybe I'm the only one who sees it that way.

Reply to this email directly or view it on GitHubhttps://github.com/FinalsClub/karmaworld/issues/278#issuecomment-36091827 .

ghost commented 10 years ago

I can give integration testing a go!

btbonval commented 10 years ago

We already have some integration tests in place thanks to @charlesconnell . Here is an example: https://github.com/FinalsClub/karmaworld/blob/96b03b44c869d46368f0332510b88c61366a514d/karmaworld/apps/courses/test/test_selenium.py#L11-L30

Charles uses Selenium to launch an instance of FireFox and then uses FireFox to interact with elements on the web page as rendered by the web server. This tests A) the client code, B) the client-server interaction, C) the server code, D) the database, E) the database-server interaction ... and all at the same time. Each test focuses on a particular use case that an end user might want to do, so integration testing tries to ensure that the expectations of user interactions are maintained.

Unit testing, by contrast, focuses on the smallest units of code and ensures that they function as expected. I think that is more powerful in mission critical systems and systems where overengineering is required. In web services, unit testing can become a rabbit hole from whence one never emerges.

I think integration testing is fantastic in web services. I'm sure we haven't tested all the end user use cases.