harvard-lil / h2o

H2O is a web app for creating and reading open educational resources, primarily in the legal field
https://opencasebook.org
GNU Affero General Public License v3.0
37 stars 30 forks source link

Integrate Celery for running async tasks #1882

Closed rebeccacremona closed 1 year ago

rebeccacremona commented 1 year ago

This PR adds Celery for the running of asynchronous tasks.

See the README for more info on how to interact with it locally. This setup is identical to the one we used with the prototyped perma-capture service and is very similar to Perma.

I included one "demo" task and scheduled it to run once a minute in dev, a) to demo config, b) to make it easy to see if your dev setup is working, and c) to have a task available for the error email test.... but.... it might not want to stay in the repo long-term, once we have at least one real task.

~I set CELERY_TASK_TIME_LIMIT to a ridiculously short 2 seconds so that we will be forced to consider what an appropriate time limit is, once we have real tasks.~

I think this gets us in good shape to start making PDFs asychronously.

Closes https://github.com/harvard-lil/h2o/issues/1798.

codecov-commenter commented 1 year ago

Codecov Report

Merging #1882 (9b308ec) into develop (b67eda4) will increase coverage by 0.03%. The diff coverage is 72.50%.

@@             Coverage Diff             @@
##           develop    #1882      +/-   ##
===========================================
+ Coverage    76.75%   76.79%   +0.03%     
===========================================
  Files           52       55       +3     
  Lines         6596     6636      +40     
===========================================
+ Hits          5063     5096      +33     
- Misses        1533     1540       +7     
Impacted Files Coverage Δ
web/fabfile.py 23.74% <9.09%> (-0.48%) :arrow_down:
web/main/tasks.py 94.11% <94.11%> (ø)
web/config/__init__.py 100.00% <100.00%> (ø)
web/config/celery.py 100.00% <100.00%> (ø)
web/conftest.py 91.45% <100.00%> (+0.05%) :arrow_up:
web/main/views.py 71.77% <100.00%> (ø)
web/main/test/functional/test_functional.py 61.11% <0.00%> (+4.29%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

bensteinberg commented 1 year ago

Staging infrastructure is set up.