Uploading process and interrupt questions

[danrademacher]

• Can the user cancel processing of a file in progress?
• Show warning if that file is large / will take a long time.
• Send link to email address when complete?

[danrademacher]

Specifically for @mojodna:

• [x] canceling Celery tasks
• [x] kill items from the database

[mojodna]

This also leads to the question of which broker to use. Ultimately it's going to be up to whoever ends up setting it up, although I'd recommend using the Redis backend (RabbitMQ is the default, but can be finicky to configure) and explicitly configuring Celery to use JSON (Python Pickle is the default until 3.2 and prevents non-Python code from easily looking at tasks) for task serialization (this makes it a bit easier to wire into unexpected tools and to have greater visibility into what's going on behind the scenes).

Using Redis provides an additional option for tracking task state and results (though there may be good Django integration if it makes sense to store the task state along with the rest of a model): http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#keeping-results

Tasks can be canceled in 2 ways (via http://celery.readthedocs.org/en/latest/faq.html#can-i-cancel-the-execution-of-a-task):

# given a result (a future)
>>> result = add.apply_async(args=[2, 2], countdown=120)
>>> result.revoke()

# given a task id
>>> from proj.celery import app
>>> app.control.revoke(task_id)

Killing items from the database is likely Django-specific, though it may tie into task states.