Optionally use mongomock instead of pymongo/mongodb

[ikondov]

Summary

Optionally replacing the mongodb database with a mocked (embedded) database can ease testing and tutorials for novice users. See this thread for more information.

We have performed an evaluation of several different solutions and have selected mongomock. A special package including the data persistence option to mongomock was created and included to the requirements.

Major changes:

• Replace pymongo.MongoClient with mongomock_persistence.MongoClient if the environment variable $MONGOMOCK_SERVERSTORE_FILE is set or the key MONGOMOCK_SERVERSTORE_FILE in FW_Config file is set. This happens centrally in fw_config.py.
• Because FireWorks runs with mongomock-persistence in a fully different environment (for example without launchpad file!) a new CI job pytest_mongomock has been added for the mongomock environment has been added. The original CI job pytest was only renamed to pytest_mongodb and not changed. All tests are selected in the pytest_mongodb job.
• The FireWorks Python API works with mongomock "out of the box", except for gridfs/filepad functionality (see comments below for details). This is why pytest markers mongodb have been added to all these tests. These 13 tests are deselected in the pytest_mongomock CI job.

Todos

If this is work in progress, what else needs to be done?

Checklist

• [x] Google format doc strings added. Check with ruff.
• [x] Type annotations included. Check with mypy.
• [x] Tests added for new features/fixes.
• [x] If applicable, new classes/functions/modules have duecredit @due.dcite decorators to reference relevant papers by DOI (example)

Tip: Install pre-commit hooks to auto-check types and linting before every commit:

pip install -U pre-commit
pre-commit install

[ikondov]

Before merging some developer should maybe have a look at this test. The latter test and this test had identical function names so that the first test has never run in the CI. And only after I changed the names it was selected by pytest but failed. This is why I marked it to be skipped.

[ikondov]

Regarding the two tests in core/tests/tests_launchpad.py that fail with mongomock: They rely on a condition when the maximum document size (16 MB) is reached to trigger storage of the FWAction object in gridfs. But this does not occur with mongomock because it does not raise the DocumentTooLarge exception. This is why I marked the test with mongodb.

[computron]

Hi, this is just a quick apology that this hasn't been reviewed yet. I promise to do so before the end of June!

[computron]

Merged - thanks for contributing this!

[ikondov]

@computron Thanks a lot for considering, reviewing and merging!