Race condition on db table creation when parallel sphinx build

[cosenal]

Describe the bug

When running a sphinx-build with the -j auto option, the build sometimes fails with the following error, suggesting a race condition when multiple processes are trying to create the same settings table.

Sphinx parallel build error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table settings already exists
[SQL: 
CREATE TABLE settings (
    pk INTEGER NOT NULL, 
    "key" VARCHAR(36) NOT NULL, 
    value JSON, 
    PRIMARY KEY (pk), 
    UNIQUE ("key")
)

]
(Background on this error at: https://sqlalche.me/e/20/e3q8)

How to Reproduce

This is not easy to reproduce, as it doesn't happen all the time (coherent with the hypothesis that it's a race condition.) The error can be observed in the following Github workflow build: https://github.com/unitaryfund/mitiq/actions/runs/9687189589/job/26731263749

Environment Information

Platform:              darwin; (macOS-14.5-arm64-arm-64bit)
Python version:        3.11.8 (main, Feb 26 2024, 15:36:12) [Clang 14.0.6 ])
Python implementation: CPython
Sphinx version:        7.3.7
Docutils version:      0.20.1
Jinja2 version:        3.1.3
Pygments version:      2.17.2

Sphinx extensions

No response

Additional context

No response

[chrisjsewell]

Heya, sphinx doesn't use sqlite, so it's gonna be to do with an extension you are using. Looking at them, it's probably myst_nb

[cosenal]

For future reference, it looks like the problem is with https://github.com/executablebooks/jupyter-cache and its db.