HIT error for celery worker

[btbonval]

[2015-01-09 19:37:22,496: ERROR/MainProcess] Task get_extract_keywords_results[37a7ed62-1b04-4564-910c-1f790ce98850] raised exception: AttributeError("type object 'HIT' has no attribute 'objects'",)
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/task/trace.py", line 233, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/task/trace.py", line 420, in __protected_call__
    return self.run(*args, **kwargs)
  File "/app/karmaworld/apps/quizzes/tasks.py", line 166, in get_extract_keywords_results
    for hit_object in HIT.objects.filter(processed=False):
AttributeError: type object 'HIT' has no attribute 'objects'

when running the celery worker by hand on beta (note: not running beat by hand, but worker).

[btbonval]

HIT model is imported from here: https://github.com/FinalsClub/karmaworld/blob/master/karmaworld/apps/quizzes/models.py#L23-L28

Looks like it ought to have a .objects attribute because it extends models.Model.

[btbonval]

Also seen on prod, and if prod works, then this error might be extraneous.

2015-01-13T18:42:03.828241+00:00 app[celerywrapper.1]: [2015-01-13 13:42:03,827: ERROR/MainProcess] Task get_extract_keywords_results[179af1f8-9f35-477e-a44a-ba0f0ee06d9c] raised exception: AttributeError("type object 'HIT' has no attribute 'objects'",)
2015-01-13T18:42:03.828254+00:00 app[celerywrapper.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/celery/task/trace.py", line 233, in trace_task
2015-01-13T18:42:03.828251+00:00 app[celerywrapper.1]: Traceback (most recent call last):
2015-01-13T18:42:03.828256+00:00 app[celerywrapper.1]:     R = retval = fun(*args, **kwargs)
2015-01-13T18:42:03.828260+00:00 app[celerywrapper.1]:     return self.run(*args, **kwargs)
2015-01-13T18:42:03.828265+00:00 app[celerywrapper.1]:     for hit_object in HIT.objects.filter(processed=False):
2015-01-13T18:42:03.828258+00:00 app[celerywrapper.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/celery/task/trace.py", line 420, in __protected_call__
2015-01-13T18:42:03.828267+00:00 app[celerywrapper.1]: AttributeError: type object 'HIT' has no attribute 'objects'
2015-01-13T18:42:03.828263+00:00 app[celerywrapper.1]:   File "/app/karmaworld/apps/quizzes/tasks.py", line 166, in get_extract_keywords_results

[btbonval]

Ah, HIT is an abstract class. Subclasses of HIT (like KeywordExtractionHIT) have .objects as do non-abstract classes which subclass models.Model (like Keyword). https://github.com/FinalsClub/karmaworld/blob/master/karmaworld/apps/quizzes/models.py#L28

>>> from karmaworld.apps.quizzes.models import HIT, Keyword, KeywordExtractionHIT
>>> HIT.objects
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: type object 'HIT' has no attribute 'objects'
>>> Keyword.objects
<django.db.models.manager.Manager object at 0x37dd6d0>
>>> KeywordExtractionHIT.objects
<django.db.models.manager.Manager object at 0x37ddd90>

Here's the problematic line: https://github.com/FinalsClub/karmaworld/blob/master/karmaworld/apps/quizzes/tasks.py#L166

It is looking for processed versus unprocessed keyword HITs in the database. Since the HIT model is abstract, it probably has no database representation. Which class is most likely supposed to be used here? Almost certainly KeywordExtractionHIT.

https://github.com/FinalsClub/karmaworld/blob/master/karmaworld/apps/quizzes/models.py#L31-L35

So does Keyword extraction work in production at all?

[btbonval]

Running on production, there are no KeywordExtractionHIT objects in the database.

>>> KeywordExtractionHIT.objects.count()
0

There are, however, Keyword objects in the database.

>>> Keyword.objects.count()
331

That is interesting, because it looks like Keywords are created in the KeywordExtractionHIT task which fails before it can create Keywords. https://github.com/FinalsClub/karmaworld/blob/master/karmaworld/apps/quizzes/tasks.py#L183

The only other way Keywords are created is through this view: https://github.com/FinalsClub/karmaworld/blob/master/karmaworld/apps/quizzes/views.py#L18

[btbonval]

• Look into the HIT sandbox to see how that works.
• Look into the MTurk HIT API to understand what's going on there

[btbonval]

Looks like this error is cleared up. Closing ticket.

[2015-01-27 13:35:27,949: INFO/MainProcess] Task karmaworld.apps.document_upload.tasks.process_raw_document[26874f9d-21d0-4708-af5f-01f32169fe1c] succeeded in 8.88806200027s: None
[2015-01-27 13:44:07,991: INFO/MainProcess] Got task from broker: get_extract_keywords_results[9d67c17d-3cc9-4b2d-af2c-065d5b0c8e9e]
[2015-01-27 13:44:08,278: INFO/MainProcess] Task get_extract_keywords_results[9d67c17d-3cc9-4b2d-af2c-065d5b0c8e9e] succeeded in 0.249867916107s: None
[2015-01-27 13:53:11,678: INFO/MainProcess] Scheduler: Sending due task check-mturk-results (get_extract_keywords_results)
[2015-01-27 13:53:11,706: INFO/MainProcess] Got task from broker: get_extract_keywords_results[342910ea-f7eb-407e-b3c8-3039a0c26ac0]

[btbonval]

fixed in commit 93d03c4688fa2e479f7cbb7e790adbbe8f913e51