HIT supposed to be optional, but breaks if missing

btbonval commented 9 years ago

I seem to recall that leaving MTURK_HOST out of the environment disables using Mechanical Turk.

However, in my environment, leaving MTURK_HOST out caused a ton of missing environment errors in the output of celery.

On the other hand, there was a pleasant little notice saying something along the lines of "Oh, I see you didn't define anything for the tweet space, so I humbly acknowledge your desire and will refrain from enacting this most twittering of tasks."

We must teach celery to be quite so respectful of MTurk!

btbonval commented 9 years ago

workaround: use the mechanical turk sandbox, just like beta does.

btbonval commented 9 years ago

hrm, it looks like the code already tries humbly acknowledge the lack of the variable and refrain from enacting the most turkiest of tasks.

I'll need to reproduce the error and paste it in full in this ticket. The code looks fine, yet I got some ugly errors instead of polite warnings.

btbonval commented 9 years ago

[2015-02-26 20:57:40,387: ERROR/MainProcess] Task get_extract_keywords_results[5f1a5b7f-fe7f-40fd-a22a-b1f159b0a209] raised exception: KeyError('MTURK_HOST',)
Traceback (most recent call last):
  File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/celery/task/trace.py", line 233, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/vagrant/karmanotes-web/local/lib/python2.7/site-packages/celery/task/trace.py", line 420, in __protected_call__
    return self.run(*args, **kwargs)
  File "/vagrant/karmaworld/apps/quizzes/tasks.py", line 158, in get_extract_keywords_results
    MTURK_HOST = os.environ['MTURK_HOST']
  File "/home/vagrant/karmanotes-web/lib/python2.7/UserDict.py", line 23, in __getitem__
    raise KeyError(key)
KeyError: 'MTURK_HOST'

on line https://github.com/FinalsClub/karmaworld/blob/master/karmaworld/apps/quizzes/tasks.py#L158

OH! except is limited to ImportError in that spot, while in another spot it isn't limited to any exception, and the actual error is KeyError. Alright, that's easy enough to fix.

Either generalize for catching any errors, or specify both ImportError and KeyError. For the latter, I wonder if there's a way to fallthrough with code to make this easier and avoid copypasta. Alternatively, set a fail switch, check if after the try..except and handle it the same regardless of which exception flipped the switch.

btbonval commented 9 years ago

Actually, ImportError would never be a problem here. It used to be that this secret was imported from a file. now that it is an env var, ImportError doesn't fit. Just KeyError. There we go!

FinalsClub / karmaworld

HIT supposed to be optional, but breaks if missing #408