search

MolSSI / QCFractal

A distributed compute and database platform for quantum chemistry.
https://molssi.github.io/QCFractal/
BSD 3-Clause "New" or "Revised" License
143 stars 47 forks source link

Replace gunicorn with waitress and fix logging issues #836

Closed bennybp closed 1 month ago

bennybp commented 1 month ago

Description

This PR replaces the embedded gunicorn instance with waitress https://docs.pylonsproject.org/projects/waitress/en/latest/#

Gunicorn was not particularly set up to be run in this way. It's largely meant to be used from the command line rather than embedded in application. Its forking model just didn't well in qcfractal and it conflicted with other multiprocessing stuff. Also, logging was a pain.

So let's try waitress instead. It is much simpler, and much more amenable to embedding in a python application. I hope this also fixes the spurious 502 Bad Gateway errors, although I wouldn't be surprised if it doesn't.

Waitress only works with one process, so the num_workers parameter in the config is no longer used.

Also as part of this PR, logging will work through a queue. This will help in the multi-process environment, and allow for using TimedRotatingFileHandler and handlers from python.

Lastly, this also quiets some logging when doing admin stuff from the CLI (like modifying users). It enables using -v and -vv to increase verbosity

Changelog description

Replace gunicorn with waitress and fix logging issues Allow increasing verbosity on the CLI with -v and -vv on the cli

Status

bennybp commented 1 month ago

Tested on the production server and everything seems reasonable